A Killed Test Run Never Reports What It Created

The zero-left assertion failed on the first rerun. The count query at the end of the parity suite returned four leftover rows instead of zero, and all four had ids starting with e2e-parity- with timestamps older than the current run. The suite runs against a regional government portal: it creates records through the information and agenda endpoints, tags every id with that prefix, and asserts at the end that no tagged row survives.

Those rows belonged to an earlier run that was killed with Ctrl-C in the middle of the suite. Its teardown never executed, so nothing removed what it had inserted.

The first guess was wrong. Collecting the created row ids during the tests and deleting that list in afterAll looked sufficient, and it does work for any run that finishes. Playwright runs afterAll once per worker after all the tests [10], and when a test fails the whole worker process along with the browser is discarded so a fresh worker runs its hooks again [11]. Neither path covers a kill from outside. A SIGKILL or Ctrl-C stops the process before any teardown runs, and the createdIds list dies with the process. Whatever the killed run inserted becomes invisible to every later run's cleanup.

So an identity net based on a run's own id list can never be complete. The only net that catches rows from a dead run is a pattern net: sweep every id matching the suite prefix straight from the database, no matter which run created it.

Sweep by pattern, verify by count

The ordering contract makes the leftovers worse than mere clutter. The listing endpoint sorts by event_date descending with NULL values last and caps the page at 9 rows, as the parity findings sheet records. Dated leftovers from dead runs occupy that page, crowd out the records the test actually created, and break both the zero-left assertion and the page-shape assertions on the next run. Two fix pieces follow from that.

First, the teardown sweeps by prefix and honours the delete contract. The portal's delete contract is two-step on the same endpoint: a plain DELETE moves the row to trash, then the same DELETE with the permanent flag removes it for good; a permanent delete without a prior trash returns 404, so a one-shot delete silently fails and leaves rows behind.

// tests/e2e/teardown.ts
test.afterAll(async ({ request }) => {
  const rows = await db.query(
    `SELECT id FROM agenda WHERE id LIKE 'e2e-parity-%'`
  );

  for (const row of rows) {
    // delete contract, two steps: trash first
    await request.delete(`/api/agenda/${row.id}`);
    await request.delete(`/api/agenda/${row.id}?permanent=1`);
  }
});

Second, the agenda sched test seeds event_date to today instead of leaving it NULL. With NULL-last ordering, an undated record sits at the tail of a limit-9 page and can fall off the visible window entirely; seeding today keeps it inside the rows the assertion inspects.

Verification is one query after the rerun. It must print 0:

SELECT count(*) FROM agenda WHERE id LIKE 'e2e-parity-%';

After both changes the rerun passed clean: the information suite ran 7/7 in 2.9 minutes, the agenda suite ran 3/3 in 51.8 seconds, and the count query returned 0.

There is one trade-off to keep in mind. A prefix sweep deletes matching rows of other runs that are still executing. Earlier testing of this pattern verified exactly that regression: a sweep wiped records belonging to a concurrent test run. The mitigation here is a narrow prefix limited to the parity suite plus a rule of no concurrent runs of these suites. Under fullyParallel or parallel CI jobs, the sweep needs a boundary clause as well, a RUN_STARTED_AT cutoff, so it only removes rows created before the current run started. The prefix is now the contract for test data, and the count query is the proof the contract held.

Sources:

[9] Playwright Best Practices

[10] test.test.afterAll – Playwright API

[11] Retries – Playwright (worker discard)