Generate Unit Tests With AI (Without Tests That Lie)

AI can produce a convincing test suite in seconds, including descriptive names, tidy fixtures and green ticks. None of those signals proves that the suite would catch a defect. If the model reads an implementation and repeats its assumptions in assertions, you have automated agreement rather than independent verification.

A useful workflow gives AI a narrower job: translate a reviewed behavioural contract into executable examples, then challenge those examples. Keep humans responsible for what should happen, and use automation to explore where that expectation might fail.

Establish an independent contract

Begin with observable behaviour, not a request to cover every line. For a discount function, specify eligible customers, rounding rules, currency units and invalid inputs. Obtain those rules from accepted requirements or a domain owner. When documentation conflicts with production behaviour, record the disagreement rather than asking the model to choose silently.

Write a compact decision matrix in plain language. An eligible order above the threshold receives a discount; an ineligible order does not; a negative subtotal is rejected. Include equality at the threshold because boundary mistakes often survive broad happy-path examples. Decide whether rejection means throwing, returning an error object or another explicit outcome.

Use the AI coding assistants directory to compare tools that can read relevant tests and conventions without requiring unrestricted repository access. Agent autonomy is less important here than clear diffs and a repeatable runner. Existing project conventions should guide syntax, but they must not become evidence that existing behaviour is correct.

Give the model examples with reasons

Provide the public function signature, agreed rules and two or three hand-calculated examples. Ask for proposed cases before executable code. This exposes misunderstandings cheaply: reviewing a sentence about a refund is easier than discovering the same error buried in a fixture builder.

Require independent expected values. A test that calculates its expectation using the same rounding helper as the implementation can preserve the same bug. Prefer literal amounts for small examples and explain the arithmetic in a comment when necessary. For larger domains, consider properties whose truth does not depend on the implementation algorithm.

A focused prompt might look like this:

Generate Vitest cases for discount(totalCents, eligible).
Contract: reject negative totals; apply 10% only when eligible
and totalCents >= 10000; round the discount down to whole cents.
First list cases and explain each expected result.
Do not import production helpers to calculate expectations.
Flag any missing rule instead of inventing one.

Use Vitest's assertion documentation to check matcher semantics. In particular, verify whether equality, floating-point comparison and exception assertions express the rule you actually intend. Familiar-looking matchers can hide surprisingly weak checks.

Choose seams without mocking away the problem

A unit boundary should separate the behaviour under test from slow or nondeterministic dependencies. Inject a clock when expiry matters, and control a network client when testing response handling. Do not mock the very transformation whose correctness the test is supposed to establish.

Distinguish a stubbed response from an assertion about interaction. Returning a fake customer record helps exercise eligibility; asserting an exact sequence of private helper calls usually couples the suite to implementation details. Prefer checking the result unless ordering itself is part of the public contract, such as charging only after authorisation succeeds.

For asynchronous work, require the model to await completion. A test may pass before a rejected promise reaches its assertion. Check that callbacks, timers and cleanup execute under the runner's intended lifecycle. The Node.js test runner documentation offers useful explanations of asynchronous tests and mocking even when your project uses another runner.

Reset mutable fixtures between cases. A shared object modified by one test can make another pass accidentally, especially when the generated suite only runs successfully in its original order. Run the new file alone and with neighbouring tests before accepting it.

Attack the assertions, not just the implementation

After the first green run, perform deliberate fault injection on a disposable branch. Change greater-than-or-equal to greater-than, remove eligibility checking, or reverse an expiry comparison. The relevant test should fail for an understandable reason. Restore each mutation before moving on, and never commit the deliberately broken code.

This is a small manual version of mutation testing. Stryker's documentation explains how automated mutation testing measures whether tests detect introduced changes. A surviving mutant is a review signal, not automatic proof of a missing requirement: some changes are equivalent under the supported input domain.

For validation helpers, use the regex tester to exercise a generated pattern against accepted and rejected strings before embedding those fixtures in unit tests. Record the runtime and flags as well as the pattern; a browser demonstration does not guarantee identical behaviour in every engine. The companion guide to proving AI-written regular expressions develops this boundary-testing approach further.

Ask AI to identify weak assertions in its own output, then review that critique independently. Assertions such as “is defined”, “has length greater than zero” or “does not throw” can be correct, but they rarely establish the full contract by themselves.

Keep the suite useful during change

Treat coverage as a map, not a verdict. An executed branch can still produce an unchecked result. Review which requirements have evidence, which error paths remain untested and which cases merely repeat the same behaviour with different inputs. Ten redundant examples are not stronger than three carefully chosen boundaries.

Avoid oversized snapshots for business rules. A snapshot can preserve accidental formatting and make review feel mechanical. Assert meaningful fields directly, reserving snapshots for stable structures where reviewers can realistically inspect the entire change. Never approve a regenerated snapshot solely because the model says the difference is expected.

Keep regression evidence next to the fix: the smallest input that reproduced the defect, the expected outcome and the issue context when available. First demonstrate that the test fails on the defective version. Then verify that it passes after the fix, rather than merely adding a green test to already corrected code.

During legacy JavaScript refactoring, label characterisation tests separately from specification tests. The former preserve observed behaviour; the latter enforce intended behaviour. This distinction lets reviewers spot an old bug that should be corrected explicitly rather than accidentally canonised. End the review by running the relevant tests yourself and recording the actual command and result.

Key takeaways

• Define expected behaviour independently before asking AI to generate tests.
• Use literal examples and explicit boundaries to challenge shared assumptions.
• Mock external dependencies without replacing the logic being verified.
• Introduce controlled faults to confirm that important assertions can fail.
• Treat coverage, snapshots and generated critiques as review aids, not proof.