# End to End Testing Without Hidden Test Order

E2E testing checks a complete user workflow through the real application boundaries that matter. A good end-to-end test starts from an explicit state, performs the actions a user would take, and verifies the outcome at the source that owns it.

That sounds simple until a suite grows.

A checkout test creates data another test quietly reuses. A permission test passes only because authentication survived from a previous case. Parallel CI moves two tests onto different workers and exposes a dependency that serial execution had hidden for months.

The browser steps may be correct. The suite is still untrustworthy if the result depends on which test ran first.

This guide explains what e2e testing should cover, how it differs from other test layers, how to choose workflows, and how to design setup, isolation, assertions, CI, and failure evidence so every test can stand on its own.

## The short answer

Build an e2e testing suite around independent product contracts:

1. Choose a small set of consequential user workflows.
2. State the user, role, data, environment, and feature flags required at the start.
3. Create that state for each test instead of inheriting it from another test.
4. Drive the browser through the behavior you actually want to verify.
5. Use locators and assertions tied to user-visible or engineering-owned contracts.
6. Verify important outcomes at the authoritative destination, not only through a success toast.
7. Run every test alone, in random order, and in parallel.
8. Preserve the first failed attempt before a retry changes the state.
9. Keep enough evidence for the receiver to identify the first contradiction.
10. Remove tests whose cost exceeds the risk they protect.

End-to-end does not mean test everything through the UI. It means exercise a complete outcome through the boundaries that make that outcome real.

## What e2e testing actually covers

An end-to-end test follows one coherent journey across the application components involved in it.

For an order approval, that path might be:

```text
sign in as an approver
→ find order ORD-4821
→ open the approval panel
→ approve the order
→ see the browser confirmation
→ read back the durable order status
→ confirm the downstream approval record exists
```

The test may cross browser rendering, frontend state, API calls, authorization, persistence, queues, and a receiving system. It should not necessarily exercise every external dependency in every run.

A useful e2e test answers four questions:

- Did the user reach the intended starting state?
- Did the product accept the intended action?
- Did the authoritative outcome persist?
- Can a receiver diagnose the first point of disagreement if it failed?

The last question matters because a long browser journey can fail in many places. A test called `order flow works` is weak if the report does not distinguish missing data, wrong role, blocked control, rejected request, stale UI, or absent downstream record.

## E2E testing versus other test layers

No single layer should carry the whole quality strategy.

### Unit tests

Unit tests isolate a function, class, hook, or module. They are fast and precise. Use them for branching logic, transformations, validation, calculations, and error handling that do not require the whole application.

### Component tests

Component tests render a UI object with controlled inputs and dependencies. They are useful for states such as empty, loading, error, permission-limited, and responsive layouts without paying the cost of a full environment.

### Integration tests

Integration tests verify that two or more components or services work together. An API handler and database, a worker and queue, or a form and validation service can often be tested more directly here than through a browser.

### End-to-end tests

E2E tests protect the small set of user journeys where wiring, state, permissions, browser behavior, and system boundaries must work together.

Use the lower layer when it can prove the same risk more cheaply and clearly. Keep the end-to-end layer for contracts that disappear when the system is split apart.

## Choose workflows by consequence

Do not begin with a goal such as “automate every manual case.” Start with business and operational risk.

Strong e2e candidates include:

- sign-in and account recovery;
- checkout, payment, refund, and cancellation;
- permission and role changes;
- release or compliance approval;
- invitation and seat management;
- file upload, processing, and download;
- issue, ticket, or message creation;
- support escalation from browser failure to engineering;
- a browser-agent action with an independently verifiable result.

Score each candidate informally by consequence, frequency, change rate, cross-system complexity, and difficulty of manual verification. Then ask whether a lower test layer can prove the risk.

A five-minute browser test for a formatting helper is a poor trade. A two-minute browser test for a refund authorization boundary may be valuable.

## Define the starting contract

The most expensive hidden assumption in an end-to-end suite is often the state before the first click.

Write it down:

```markdown
Workflow: approve order
User role: regional approver
Tenant: synthetic test workspace
Order: ORD-4821
Order status: pending approval
Permission: approve orders below $10,000
Feature flags: approval-v2 enabled
Locale: en-US
Browser project: Chromium desktop
Expected durable result: order status becomes approved
```

“Logged in” is not enough when role, tenant, permissions, data, locale, or feature flags can change behavior.

Create the required state through a stable setup API, fixture, or approved database-safe helper when possible. The browser should exercise the behavior under test, not spend most of its runtime navigating unrelated setup screens.

Use the UI for setup when the setup journey itself is part of the contract. If the test is about approving an order, creating the customer, product, and pending order through five earlier screens can add cost without adding relevant coverage.

## What our controlled test-order experiment found

For this article, the Samelogic team built a deterministic synthetic order-review page. It used no customer data and produced no external side effects.

The fixture had two browser actions:

1. **Create order** created `ORD-4821` through a local API.
2. **Open order** expected that record to exist and rendered it for review.

We ran the two tests in five disclosed orders with fresh browser contexts in Google Chrome 151.

First, the shared-state suite relied on Create order running before Open order. Then we ran the same orders with explicit setup that seeded the required record for the review test.

| Suite design | Full runs passed | Review tests passed |
|---|---:|---:|
| Shared state inherited from another test | 3 of 5 | 3 of 5 |
| Independent setup for each test | 5 of 5 | 5 of 5 |

The two shared-state failures occurred whenever Open order ran first. The test did not expose a product defect. It exposed an unstated dependency on another test.

The experiment completed in 12,616.54 milliseconds across twenty browser test observations. We retained the complete run matrix, environment metadata, two screenshots, file sizes, and SHA-256 checksums in the August 27 publication artifact.

The conclusion is narrow:

> A test that passes only after another test creates its state is not independently repeatable. Explicit per-test setup removed the order dependency in this synthetic fixture.

This is not a benchmark of test frameworks or a production flake-rate claim. It is a controlled demonstration of why serial green runs do not prove isolation.

## Make each test independently repeatable

The official [Playwright browser-context documentation](https://playwright.dev/docs/browser-contexts) describes isolation through a fresh browser context for every test. That gives each case separate cookies, local storage, session storage, and other browser state.

Browser isolation does not automatically isolate backend data.

A fresh context can still connect to the same account, order, workspace, queue, or database records as another test. You need both:

```text
browser isolation
+
application-data isolation
```

Practical patterns include:

- create a unique user or record for each test;
- use worker-scoped accounts only when tests cannot mutate overlapping state;
- generate stable unique IDs from worker and test identity;
- make setup and cleanup idempotent;
- reset feature flags explicitly;
- avoid shared “latest order” or “default customer” records;
- make retries start from a known state rather than the failed attempt's leftovers.

If cleanup fails, the next run should still be able to create or recover its own state safely.

## Run tests in random order and alone

A suite can look healthy because the runner uses the same file order every time.

Add three checks to CI or a scheduled reliability job:

1. Run each important test alone.
2. Shuffle order with a recorded seed.
3. Run with more than one worker.

When a failure appears only under one order, preserve:

```markdown
Randomization seed:
Test order:
Worker count:
Account and data IDs:
Starting state:
First test that mutated shared state:
First contradictory assertion:
Retry result:
```

The seed turns a random failure back into a reproducible run. Without it, randomization creates evidence you cannot replay.

## Use assertions that match the real outcome

A browser message can be the correct assertion for a local interaction. It is not always the authority for a consequential mutation.

Weak completion contract:

```ts
await page.getByRole('button', { name: 'Approve order' }).click();
await expect(page.getByRole('status')).toHaveText('Approved');
```

Stronger when the backend owns approval state:

```ts
await page.getByRole('button', { name: 'Approve order' }).click();
await expect(page.getByRole('status')).toHaveText('Approved');

const order = await request.get(`/api/orders/${orderId}`);
expect((await order.json()).status).toBe('approved');
```

Add a downstream assertion only when the downstream object is part of the user promise. If approval must create a Jira issue, verify the canonical issue. If it only updates the order, do not add unrelated integrations to make the test look comprehensive.

The rule is simple: assert each material outcome at the system that owns it.

## Keep locators aligned with maintained identity

E2E testing fails noisily when selectors depend on incidental DOM structure.

Prefer a role, label, visible identity, scoped relationship, or explicit test ID that the team intentionally maintains.

```ts
const order = page.getByRole('row', { name: /ORD-4821/ });
await order.getByRole('button', { name: 'Review' }).click();
```

Avoid hiding ambiguity with `.first()` or positional selectors. A strictness failure may be telling you that the page exposes duplicate names, the test omitted a meaningful scope, or a hidden control remains active in the DOM.

The locator is part of the product contract. A failure should point toward a changed user or engineering identity, not a wrapper added during a refactor.

## Control dependencies without deleting reality

A full end-to-end suite that calls every real third party can become slow, expensive, and unstable. A suite that mocks every boundary can pass while production wiring is broken.

Use layers:

- deterministic responses for unrelated third parties in the main suite;
- contract tests for request and response schemas;
- a small integration suite against real sandboxes;
- synthetic monitoring for production-safe critical paths;
- explicit failure tests for timeout, denial, retry, and partial completion.

For payment, email, maps, analytics, or identity providers, decide which runs need the real service. Record that decision instead of allowing environment availability to choose for you.

## Make parallel CI a design constraint

Parallel execution is not only a speed feature. It is a test of whether your suite has hidden shared state.

Before increasing workers, inspect:

- accounts reused by several tests;
- fixed record names and IDs;
- shared downloads and output directories;
- mutable feature flags;
- one-time tokens;
- rate limits;
- queue consumers;
- test cleanup that deletes another worker's data;
- external sandboxes that serialize operations.

Give each worker a namespace. Keep it in logs, screenshots, traces, records, and receiving-workflow links.

```text
run-20260827-worker-03-order-4821
```

When a collision happens, the receiver can see which worker created and changed the object.

## Preserve the first useful failure

Retries are useful for measuring instability. They are dangerous when the retry becomes the only artifact anyone sees.

A retry can change:

- browser process and storage;
- account or data state;
- worker order;
- cache warmth;
- deployment version;
- timing and network path;
- external service response;
- feature-flag evaluation.

Preserve the first attempt before retrying. The [Playwright best-practices guide](https://playwright.dev/docs/best-practices) recommends configuring debugging artifacts such as traces, and its broader isolation guidance keeps failures easier to interpret.

For an expensive browser workflow, retain:

```markdown
- test title and expected outcome
- build, browser project, worker, role, and data identity
- known-good starting state
- action path to the first contradiction
- focused console and network evidence
- screenshot when the visual state matters
- trace for timing or stateful failures
- authoritative readback result
- canonical CI or receiving-workflow URL
```

Do not collect everything by default. Preserve the smallest evidence set that helps the receiver decide what failed.

## Learn from realistic practitioner workflows

A recent [Ministry of Testing discussion](https://club.ministryoftesting.com/t/javascript-typescript-vs-python-for-test-automation-and-more/87677) moved from language choice to a more useful practice project: create data through an API, exercise a Playwright checkout flow, verify the backend result, introduce fixtures, run parallel CI, and retain a trace on failure.

That sequence exposes the real engineering work in e2e testing.

The hard problems are not typing `click()` and `fill()`. They are choosing state, controlling ownership, designing assertions, surviving concurrency, and leaving evidence another person can use.

## A review card you can copy

```markdown
E2E test review

Workflow and user outcome:
Why this requires end-to-end coverage:
User, role, tenant, and permissions:
Required starting data:
Setup owner and method:
Browser project and environment:
Action path:
Visible assertion:
Authoritative outcome assertion:
Downstream destination, if required:
Shared-state risks:
Random-order result and seed:
Parallel-worker result:
First-attempt evidence retained:
Retry policy:
Expected runtime and maintenance owner:
Removal or downgrade condition:
```

Use this card for important or unstable workflows. Routine tests do not need paperwork for its own sake. The goal is to make hidden contracts visible before CI discovers them.

## Common mistakes

### Turning every case into a browser journey

Keep logic, validation, and component states at cheaper layers when they do not need the complete system.

### Reusing one account because setup is slow

Fix setup or namespace the data. A shared mutable account often turns speed savings into order-dependent failures.

### Asserting only the success toast

Verify durable state when the product promise extends beyond the browser message.

### Running only in one fixed order

A deterministic order can preserve hidden dependencies indefinitely.

### Increasing retries without preserving the first attempt

A pass on retry is evidence of changed conditions, not automatic proof that the original failure was harmless.

### Collecting giant traces without a receiver contract

Capture enough to identify the starting state, action path, first contradiction, and outcome. More data is not always more useful.

### Measuring only pass rate

Track first-attempt pass rate, order-dependent failures, isolated-test failures, retry recovery, median runtime, diagnosis time, and repeat failures by workflow.

## The practical rule

Good e2e testing protects a complete outcome without making every test depend on the history around it.

Choose consequential workflows. State the starting contract. Create independent data. Drive the real browser behavior. Verify the result at the source that owns it. Run alone, shuffled, and in parallel. Preserve the first failed attempt before a retry changes the scene.

Samelogic's [flaky Playwright tests workflow](https://samelogic.com/workflows/flaky-playwright-tests) supports the adjacent receiver problem when an original browser failure disappears after rerun. A QA practitioner or permitted operator deliberately captures the browser path and relevant state as a replayable test artifact. This article owns the distinct implementation intent of designing independent end-to-end tests rather than duplicating that product workflow.

## Sources

1. Playwright best practices: https://playwright.dev/docs/best-practices
2. Playwright browser-context isolation: https://playwright.dev/docs/browser-contexts
3. Ministry of Testing discussion about realistic Playwright project work: https://club.ministryoftesting.com/t/javascript-typescript-vs-python-for-test-automation-and-more/87677
4. Samelogic flaky Playwright tests workflow: https://samelogic.com/workflows/flaky-playwright-tests