Tech Notes

How To Structure Test Cases So AI Can Actually Automate Them

AI tools break on messy test cases differently than humans do — they don't ask clarifying questions, they guess. A one-line goal, reusable preconditions, named test data, and explicit assertions fix that.

Aleksandra Obradović··5 min read
Hotel, 1955, oil painting by Slovene painter Melita Vovk.
Hotel, 1955, oil on canvas, by Melita Vovk (1928–2020), Slovene academic painter, illustrator of over 80 children's books, and co-founder of Grupa 53. Melita Vovk, via Pilonova Galerija.

TL;DR: Handing test cases to an AI tool to generate or review automated test code is now a normal part of the QA workflow. A messy test case still breaks automation, but the failure mode has changed: an AI assistant does not ask clarifying questions the way a human engineer might. It assumes, fills the gap, and moves on. The fix has not changed: write a clear one-line goal, reusable preconditions, named test data, and explicit assertions. It matters more now that the automation side talks back less than it used to.

If a test case reads like a novel, nobody wants to automate it. These days, “nobody” often means the AI tool sitting between the test case and the automated test it produces. Most manual-style test cases are written for a human who already knows the app. Instructions like “tap the button” or “you should see the list” never pin down which button, which list, or what state the app needs to be in first. A human tester fills those gaps from context. An AI assistant fills them too, with an assumption instead of a question, and not always the right one.

Principles for automation-friendly test cases

Every test case needs four things: a clear goal, stable preconditions, reusable data, and explicit assertions.

Avoid copying the same setup steps into ten different cases. Avoid mixing setup and verification in the same step. Avoid expectations vague enough that “pass” becomes a judgment call.

Before / after: the reimbursement status filter

Field Before (manual-only) After (automation-ready)
Title “Filter reimbursements by ‘Missing Requirement’” “Reimbursements: filter by Missing Requirement shows only matching requests”
Preconditions User is logged in. User is authenticated as user_with_mixed_reimbursements. Test data includes at least one reimbursement with a missing requirement and one fully ready reimbursement.
Steps Open the app → Go to Reimbursements → Tap Filter → Tap “Missing Requirement” → Tap Apply. Navigate to Reimbursements list → Open filter panel → Select filter “Missing Requirement” → Apply filters.
Expected result Only reimbursements with missing requirements are shown. Only reimbursements where status = MISSING_REQUIREMENT are visible. No reimbursements where status = READY are visible.

The before version is readable. It breaks the moment anyone, human or AI, tries to automate it. “User is logged in” as who, with what data? “Only reimbursements with missing requirements are shown,” checked how, against what? The after version answers both questions before they are asked. It names the user, names the data states, and gives a status field to assert against instead of a screenshot to eyeball.

From manual case to automated test

Manual structure Automation equivalent
Preconditions Fixtures / factories
Steps Framework commands (tap, scroll, wait, expect)
Expected results Explicit assertions on specific IDs or state

This mapping holds regardless of which framework sits underneath it. Automation-first is not required from day one. Write manual cases so they do not have to be thrown out once automation catches up.

Why this matters more, not less, with AI in the loop

It is tempting to assume AI makes a sloppy test case less costly. The tool is fast, it fills gaps, close enough. In practice the opposite is true. AI defaults to assumption, especially under ambiguity, and unlike a human automation engineer, it usually will not stop and ask. It picks an interpretation, generates code against it, and the test starts failing for the wrong reason before anyone notices the guess was wrong.

An automation-ready structure removes exactly the ambiguity AI would otherwise fill in on its own. Pair it with an instruction to follow the existing test style already in the codebase instead of inventing a new pattern. Better still, codify that standard once as a reusable skill the AI checks against instead of restating it in every prompt. Then it can work largely unattended: less back and forth, fewer clarifying questions, fewer silent wrong guesses that surface three sprints later as a flaky test nobody trusts.

Structuring suites, not just individual cases

The same discipline scales up. Group tests by feature. Reimbursement filter tests, for example, get grouped into one suite covering statuses, the clear button, and multiple selection, sharing preconditions and navigation steps instead of each case re-deriving them from scratch. Keep naming aligned between test case titles and the corresponding automated test files, so moving between the two does not require translating names in your head.

Checklist before calling a case “automation-ready”

  • Title describes behavior, not just UI.
  • Preconditions are explicit and reusable.
  • At least one assertion is on data or state, not purely visuals.
  • No duplicated “open app / login” steps across cases.
  • Test data (user, account state) is specifically named, not implied.

Bottom line

Automation-readiness is not a separate pass tacked on after writing tests. It is a small set of habits: name the data, state the goal in one line, assert on state. These habits made cases easier to hand off to a human automation engineer. They now make cases easier to hand off to an AI one, for the same reason: they leave less for someone, or something, else to guess.

When to skip this?

Skip this structure for a one-off exploratory test that will never run again. Write it fast and throw it away. The structure earns its cost specifically for regression-suite cases: the ones that get read, run, and eventually automated by someone, or something, other than the person who wrote them.

Tags#QA#ai
Share
Written by
Aleksandra Obradović

Curious about technology, product thinking, and anything worth reading.

Continue reading

A quieter inbox.

Sent occasionally, whenever there's something worth sharing — new essays, things worth reading, no fixed schedule.

Free. Unsubscribe in one click.