For QA leads and release managers who decide if a version can ship.
Your setup
- Staging runs the release candidate, for example version
2.5.0. - Certyn suites run there, and maybe your CI pushes its own results.
- Your deploy sets the environment's version, so runs and issues know which version they belong to.
- You want one answer with the reasons.
Say this to Certyn
Can we ship staging? Check 2.5.0 against our Definition of Ready and list the blockers.
In the web chat, the /release-readiness command types a similar request for you.
What Certyn does
- Reads the version on staging and its deploy history (
app.environment.get). - Gets the release summary for that version (
app.release.get): its runs, its issues, open issues from earlier releases, and a verdict:- not ready: an open critical or major issue found in this release, or a failed or blocked run on it;
- not tested: no finished runs on this version;
- ready with risks: only open minor or trivial issues, or open issues from earlier releases that were not checked again;
- ready: no known blockers.
- Reads your Definition of Ready in the wiki (Testing Rules) and checks its other rules: pass rate (
app.metrics.get), flaky tests, Needs Review items, and Sentry errors if Sentry is connected. - Answers go or no-go, with each blocker linked to its issue or run, and what was not tested.
- If the regression suite has not run on this version, says so and offers to start it. It asks before it starts a run.
What you get
- A go or no-go, with ranked blockers and links.
- Warnings that need a sign-off, for example only flaky tests failing.
- What is missing: suites that did not run, CI results that did not arrive.
Limits
- The verdict uses only known evidence: what ran on this version. It is not a guarantee. Areas that no run covered are unknown, not passed.
- Every new project's wiki starts with a default Definition of Ready (no open critical or major bugs, pass rate at least 95%, regression run in the last 24 hours). Edit it to your team's rules.
- No version on the environment means runs and issues have no version, and there is no release to judge. Set the version from your deploy.
- A deploy of
1.0.0,v1.0.0or a full 40-character commit SHA does not create a release by itself. Use real version names, or ask Certyn to create the release.