The Route Smoke Test That Stopped at Status 200

That night the L1 crawl test in my admin repo passed everything, every route green. Then I opened three pages by hand and all that showed up was an empty shell: header there, sidebar there, page content missing. The test still called it a pass because each route did return 200.

My wrong guess sat exactly there. For a long time I carried a broken rule of thumb: if the route returns 200, the page must be alive. In reality 200 only proves the server sent something, not that React managed to render anything. An empty shell is a successful 200.

The fix was not adding more tests, but changing the question. Each route is no longer asked whether it responds, but whether its key element shows up. I call it a key-element map, and the format is simple: every static admin route gets exactly one element that proves the page has rendered its content. It can be a table testid, a form label, a heading. Locator-wise, the component at the center of Playwright's auto-waiting machinery [5]. One element, one route, no exceptions.

One key element per route

The map is an object: the key is the route path, the value is one Playwright locator.

// tests/l1.routes.spec.ts
import { test, expect, type Locator } from '@playwright/test';

const keyElements: Record<string, Locator> = {
  '/admin/orders': page.getByTestId('orders-table'),
  '/admin/settings': page.getByLabel('Nama toko'),
  '/admin/reports': page.getByRole('heading', { name: 'Laporan' }),
  '/admin/promos': page.getByLabel('Buat promo')
    .or(page.getByTestId('promo-empty')),
};

for (const [route, key] of Object.entries(keyElements)) {
  test(`L1 render ${route}`, async ({ page }) => {
    test.setTimeout(45_000);
    await page.goto(route);
    await expect(key.first()).toBeAttached();
  });
}

Those last three lines are the heart of the test. Every decision that keeps this test alive lives there, and unfortunately I found them through two consecutive failures on the first run.

The first two failures that turned out useful

The first failure was embarrassing: I wrote await locator.toBeAttached() directly on the locator. That is not an API, so the run errored out before checking anything. Playwright works with a web-first assertion model: the auto-retrying web assertions wait until the expected condition is met, and they must go through expect(...), not be called as locator methods [9]. The correct form is await expect(locator).toBeAttached(). The difference is not styling: through expect, the assertion re-fetches the element over and over until the condition is met or the timeout runs out [9]; calling a method that does not exist only hands you a TypeError.

The second failure was more interesting. I first gave a dropdown route a key element shaped like one option's testid. The test kept failing, even though the option clearly existed when I checked by hand. It turned out the option's testid only renders when the dropdown is open. In the closed state, the DOM simply does not contain it.

There I wrote a rule I no longer break: the key element must exist in every render state. An empty state still counts as a valid render state. A closed dropdown does not.

So for that route, the key element moved to the aria-label on the closed trigger, because that attribute exists from the first moment of render. For routes that have both an empty and a filled state, I use locator.or(), which merges two locators into one [8], like the promo route in the example: the "Create promo" label for the empty state, the testid for the filled one. If either shows up, the route passes.

Two details that keep the test from being fragile

Almost every admin page in this project is responsive, and the render pattern is two DOMs at once: one for desktop, one for mobile, one of them always hidden. If the assertion were toBeVisible(), the test would fail for the version that is not shown in the test viewport. Since all I need is proof the page rendered, toBeAttached() is enough, that assertion only requires the element to be attached to the DOM without demanding visibility [9], and .first() handles the duplicated DOM.

On timing: the dev server on the CI machine still compiles brand-new routes on first request, and that can exceed 15 seconds per route nobody has visited yet. The default assertion timeout is too tight for that. Because each test file only runs one route, I raise it to 45 seconds via test.setTimeout() inside the test, not globally, so routes that are genuinely fast do not inherit the generous tolerance.

The quick sanity check: run the test against one route whose key element you deliberately removed from the component. The test must fail, and the message must name that locator. If the test still passes, that element was never really the key.

Some people say a level-one smoke test is too shallow to bother improving and you should jump straight to L2. I disagree. L1 is the cheapest layer that still proves rendering, and its cost per route is one goto plus one assertion. Letting L1 pass on empty shells wastes every layer above it on something that was broken from the start.

The bonus is not small either: a key-element map is documentation that cannot go stale about what each page is for. If a new contributor asks what makes the promo page count as "alive", the answer lives in the spec, not in an old-timer's memory. If the page changes purpose one day, the test fails and forces the map to be updated. That is what I want from a cheap test.

Sources

5. Locators - Playwright 8. Locator - Playwright 9. Assertions - Playwright