For QA leads who want to know which parts of the app no test checks, and want tests for them.
Your setup
- A web app with a test environment that Certyn can sign in to.
- Some Certyn test cases already, or none yet.
- You want a list of gaps by area, then test cases for the gaps you pick.
Say this to Certyn
Which areas of our app have no tests? Show the gaps, then write test cases for the ones I pick.
What Certyn does
- Reads the coverage map (
app.coverage.get): areas, and for each tested behavior its status:- covered: has active tests and the last run did not fail;
- degraded: the last run failed or was blocked, or all its tests are quarantined;
- proposed: all its tests still need review;
- uncovered: all its tests are obsolete. It also flags behaviors not run in the last 14 days.
- Compares the map with what it knows of the app (the wiki, its memory, pages it has tested) and lists areas with no test cases at all.
- For each gap you pick, checks the flow in a browser: on the chat's Machine, or in a short explore run with that goal. It asks before it starts a run, which uses plan time.
- Writes test cases from what it ran (
app.testcase.create): steps, expected results, tags (defaultregression) and onecoverage_keyper behavior, for examplecheckout/apply-coupon. It searches first, so it does not add duplicates. - Reads the map again and shows what changed.
What you get
- A gap list by area: no tests, failing, not reviewed, not run lately.
- New test cases that join every suite with a matching tag.
Limits
- The map lists only behaviors that have test cases with a
coverage_key. An area with no tests is not in it. Certyn finds those gaps from the wiki, memory and exploring, so the list is only as good as what it knows. Write your main product areas in the wiki for a better list. - Old test cases without a
coverage_keydo not count. Ask Certyn to add keys to them (app.testcase.update). - There is no coverage page in the app. Ask in the chat.
- Status uses runs on one environment, the chat's environment by default.
- New test cases are active at once. To check them first, ask Certyn to mark them Needs Review. They then skip suite runs until you approve them on the Tests page.