Skip to content
RecipesDoc

Explore a New Feature Before Release, Get Bugs and Test Cases

Certyn explores a new feature on staging, files the bugs it finds with evidence, and saves test cases for the flows it verified.

For teams with a new feature on staging and no tests for it yet.

Your setup

  • A new feature is deployed to staging, for example bulk edit on /products.
  • A test account that can use the feature.
  • The release is in a few days.
  • You want the bugs now, and test cases to run again later.

Say this to Certyn

We shipped bulk edit on /products to staging. Explore it for 30 minutes, file the bugs, and write test cases for what you verified, tagged bulk-edit.

What Certyn does

  1. If no test account is saved, opens the sign-in form (sign_in).
  2. Says the run length and starts an explore run with your goal (app.run.create type explore, minutes 30). A run uses plan time while it runs.
  3. In the run, Certyn reads the wiki, memory and past issues, then writes a checklist with the riskiest flows first: saves, deletes, access, pages full of data.
  4. It tests each line. For each confirmed bug it first searches for a matching issue, then files a new one with screenshots, steps, expected and actual result.
  5. Because you asked for test cases, it saves one for each real flow it ran and verified (app.testcase.create), with your tags. It updates a matching case instead of adding a duplicate.
  6. It writes the run report: verdict, what it covered and what it did not.

What you get

  • One Certyn issue per bug, in review. Confirm or close each one.
  • Test cases tagged bulk-edit, with steps and expected results from what Certyn saw.
  • A run report with the areas it did not cover.
  • If you ask: a suite of the bulk-edit cases that runs after each deploy.

Limits

  • Test cases come only from flows Certyn ran and verified, not from every possible path. Read the "not covered" list and ask for a second run on those areas.
  • A test case gets the regression tag when you name no tags, so it joins the nightly Regression Suite.
  • Certyn judges against your request, the wiki and any spec. Without a spec, a missing feature is noted, not filed as a bug. Paste or attach the spec to get checks against it.
  • A feature behind a flag must be on for the test account.
  • A run has 60 minutes by default, up to 240.