We Stopped Writing E2E Tests Against the UI

The title is a deliberate provocation, so let me say up front what it does not mean. We did not stop driving the UI. Playwright still opens a real browser, logs in, fills forms and reads what comes back. What we stopped doing is letting one temporary implementation of the UI be the language the tests are written in.

Here is the kind of spec I mean, lightly anonymised from something we inherited:

Click the Add button. 

Select the second option in the dropdown.

Fill the third input with the value.

Click Save in the modal footer.

Expect the row to appear in the table.

There is nothing wrong with any of those steps on the day they were written. The problem is that this is a description of how one version of a page happened to be assembled. It says nothing about why the user is there or what the product promised them. Six months later the modal is a full page, the dropdown is a segmented control and the save button lives in a sticky footer. The product makes exactly the same promise it did before and the spec is dead.

We went through precisely that redesign on a rule-management area of a product, and it was the thing that pushed us from "we should probably abstract this" to actually restructuring the suite. In the first post in this series, implementation coupling was the fifth and most expensive cause of instability on the list; this post is the long version of the fix. What follows is the structure we landed on and, more importantly, the rule we use to decide which layer a change belongs in.

The behaviour is the stable contract

The same workflow, written as a procedure:

Given a business rule is configured

When an authorized user edits the rule

Then the rule reflects the updated configuration

No modal, no tab, no button label, no component library, no field order, no route. What it does commit to is a contract the product has actually made: an authorised user can initiate editing, a valid configuration can be submitted, the system persists and displays the updated rule, and the change is visible through whatever downstream behaviour depends on it. As long as that contract holds, the procedure is still a true statement about the product and the test should still pass.

I have kept the example generic on purpose. Swap "business rule" for notification policy, approval rule, access setting, risk threshold, pricing condition or routing rule and the shape is identical: a thing with a stable product meaning and an interface that will change under it.

Five layers, each with one job

We ended up with five responsibilities, and the discipline is in keeping them from leaking into each other.

The behaviour procedure is the spec itself, in Given/When/Then form. Product, QA and domain people can review it without knowing Playwright or the component tree, and that review is where most of its value comes from.

The keyword contract is a controlled vocabulary: "the user creates a valid business rule", "the user searches for an accessible record", "the rule is active in the configuration list". Each keyword has a documented intent, a scope, an expected result and an evidence requirement. This is the layer people underestimate. A keyword is not a friendly alias for page.click(); it is a maintained adapter between a product capability and whatever its current UI realisation happens to be.

The domain action coordinates the workflow. It may set up state through an API, resolve a data example, call into several screen components, handle supported variants and register cleanup. It knows what the operation means. It deliberately does not know where any field currently sits.

The screen binding knows the present shape of the interface: fields, confirmation dialogs, navigation, loading states, validation messages, submission behaviour. This is where the redesign lands.

The locator contract is the bottom: stable data-testid attributes, accessible roles, and the small number of hand-chosen selectors that connect a binding to rendered components. Browser mechanics stop here. They do not travel upward.

What the redesign actually changed

The old flow was a table row, an edit icon, three fields in a modal and a save button. The new one was a details page with a side panel, progressive disclosure, and a confirmation step before anything persisted. Field locations, component structure and the user's path through the screen all changed.

The product capability did not. An authorised user could still create and maintain a rule. So the keyword stayed in every procedure that used it, and the action and screen bindings underneath it were rewritten for the new experience and new locators. Every functional, permissions, security and data-driven scenario that used the keyword picked up the new implementation through one repair. Most of the automation diff for that redesign sat in the shared action and screen layers, and the procedure files barely moved.

That is the leverage, and it is worth being precise about where it comes from. It is not that the abstraction is clever. It is that the blast radius of a UI change is bounded by the number of bindings, not the number of specs.

Stable test IDs are necessary and not sufficient

Every time this topic comes up someone says "we solved this with data-testid". Stable test IDs are a real improvement: they insulate locators from CSS, layout and copy changes, and they are a good interface between frontend developers and automation. We require them.

They do not solve duplicated workflow knowledge. If twenty specs each contain the same five test IDs and the same click sequence, a redesign still means twenty repairs. The selectors were stable; the implementation was copied. Test IDs belong in the locator and screen layers. The reusable action belongs above them, and the behaviour keyword above that. Stability comes from controlling where change lands, not from picking a stronger selector.

One capability, many test purposes

The thing that convinced the sceptics on the team was watching the same keyword show up in procedures that exist for completely different reasons: ordinary rule creation, a data-driven matrix of rule types, a combined-conditions validation case, a permissions boundary, a security case feeding harmless untrusted text through the form, a representative usability flow, and a requirement-level verification protocol for the regulated layer of the product.

Those tests have nothing in common at the level of intent. They share the mechanics of creating a rule. When the binding changes, each of them keeps its own purpose and gets the common repair for free.

Keep setup out of the visible workflow

A behaviour-first procedure does not require every precondition to be created through the UI, and I would argue it requires the opposite. If a test is verifying an approval decision, spending half the script creating an organisation, assigning permissions, importing history, configuring a rule and waiting for a background job obscures the thing under test.

The layered design puts preconditions behind APIs, fixtures or simulation helpers. The Given step may navigate so that the prepared state is visible. The When step performs the behaviour being verified. The script is shorter because irrelevant choreography is gone, not because the premise is hidden; the prepared state is still named in the procedure and independently verifiable.

When the procedure has to change

Abstraction is dangerous when it hides real product change, so this rule matters more than any of the layering. The procedure changes when the user's intent changes, when a new decision or confirmation is introduced, when the observable outcome changes, when the permitted actor changes, when failure guidance changes in a way the user would notice, or when the workflow gains a legally, financially, safety or operationally significant step.

If a destructive action now requires an explicit impact confirmation, that is not a new locator. It is new behaviour, and it belongs in the procedure where a reviewer will see it. The short version we use on the team:

Refactoring changes bindings. Behaviour changes procedures.

That boundary turns out to be useful for change control as well as maintenance. A locator-only refactor can be reviewed as a rebinding with no behavioural impact. A changed decision, permission, confirmation or outcome triggers impact analysis and broader re-verification. In a regulated product, including SaMD, that distinction decides which requirements, risk controls, approvals and evidence sets have to be revisited. The layering does not remove that work. It makes its scope explainable, which is most of the battle in an audit.

Keywords at the right altitude

Keywords fail in two directions. Too technical and they are selectors with extra steps:

When the user clicks rule-submit

Too vague and they cannot produce a meaningful failure:

When the user manages settings

The useful version names an actor, an intent and a domain concept:

When the user creates a valid business rule

That survives a redesign and still tells you, when it fails, what the product stopped doing.

The maintenance outcome

The value of E2E abstraction is not the count of helper methods. It is whether each kind of change lands in the layer built to absorb it. With this design, component refactors change locator bindings; UX redesigns change screen and action implementations; new data variants change example matrices; genuine behaviour changes change procedures; provider changes are caught at contract boundaries; and regulated change impact can be scoped from whichever layer moved.

We still test the UI end to end. We stopped letting the UI write the tests.