Testing is one of the most practical places to apply AI in software delivery. The work is often repetitive, the backlog is always long, and the cost of gaps shows up in production.
AI tools now generate test cases from code and requirements, suggest edge cases, maintain UI test selectors and summarise failures. Used well, they widen coverage and cut maintenance. Used carelessly, they produce large suites that pass and prove very little.
Test generation
Generating tests is the most visible use. Given a function, an API spec or a user story, a model can draft unit tests, integration tests and test data.
Where it helps:
- Coverage of the obvious — happy paths and common error cases written in minutes
- Edge cases — boundary values, empty inputs, unusual formats people forget
- Test data — realistic, synthetic records that avoid copying production data
- Legacy code — characterisation tests that capture current behaviour before a refactor
Where it falls short: a model generating tests from the implementation tends to encode what the code does, not what it should do. If the code has a bug, the test may faithfully assert the bug.
Generate from requirements and specifications where possible, not only from code. Then have an engineer review what each test actually checks.
Self-healing tests
UI and end-to-end tests break for reasons unrelated to real defects: a renamed element, a moved button, a changed layout. Self-healing tools use multiple signals to locate the intended element when the original selector fails, and propose or apply a fix.
This reduces maintenance noise. It also carries a risk. A test that quietly heals itself can mask a genuine change in behaviour — a button that moved because a flow broke, not because of a redesign.
Sensible guardrails:
- Log every heal, with before and after
- Require review for heals above a set threshold or in critical flows
- Fail, rather than heal, when the change touches assertions or business logic
- Track heal frequency, since a rising rate often signals unstable UI or poor test design
Failure triage
Large suites produce a lot of red. AI can cluster failures, link them to recent changes, separate flaky tests from real regressions and draft a summary for the team.
This saves time, but it is advisory. The engineer still decides whether a failure is a defect, a test problem or an environment issue.
Testing AI systems themselves
If you build AI features, your test strategy needs another layer. Model outputs are not deterministic in the way traditional code is. That calls for:
- Evaluation sets with expected behaviours, graded by rules, reviewers or carefully validated model-based graders
- Regression runs whenever prompts, models or retrieval data change
- Adversarial cases for prompt injection, data leakage and policy bypass
- Sampling in production to catch drift that offline tests miss
The same rule applies: people decide what good looks like, and tools help check it at scale.
Where humans stay in the loop
AI can draft and maintain tests. It should not own the judgement. Keep people responsible for:
- Defining acceptance criteria and what correct behaviour means
- Reviewing generated tests for meaningful assertions
- Approving heals in critical paths
- Deciding release readiness from test and evaluation results
- Exploratory testing, where curiosity finds what scripted tests never will
The goal is fewer hours on repetitive work and more on the testing only people can do.
Watch the metrics
More tests is not the same as better testing. Useful signals include:
- Defects found before release versus after
- Mutation testing scores, which show whether tests catch injected faults
- Flaky test rate and time spent on maintenance
- Time from failure to diagnosis
If coverage climbs but escaped defects do not fall, the generated tests are not checking the right things.
Where to start
- Pick one area with a clear maintenance burden, such as a brittle UI suite
- Generate tests from specifications, and review assertions before merging
- Enable self-healing with logging and review thresholds, not silently
- Add mutation testing to measure whether the new tests catch real faults
- Keep release decisions with a named person
AI makes testing cheaper to scale. People still decide what is worth testing.