Assessing Release Readiness
A practical checklist for deciding whether an environment is ready to ship, using Certyn signals and evidence.
Before releasing, use Certyn's signals to make an evidence-based decision. This guide provides a concrete checklist you can evaluate against your project's data.
Release readiness checklist
Latest regression run completed
Run your full regression suite on the target environment. All Required and Critical priority tests must have a terminal result — no tests should be stuck in progress or queued.
No failures on Required or Critical tests
Any Required or Critical test marked Verified Failed is a release blocker. These must be fixed and re-verified before shipping.
No open Critical or High severity issues
Check for unresolved issues in the target environment. Open Critical or High severity bugs indicate significant risk.
Pass rate above threshold
Overall pass rate should be above 95% (or your project's configured threshold). A declining pass rate signals growing instability.
No unresolved Needs Review items on Required tests
Items flagged Needs Review on Required tests are ambiguous — they could be hiding real failures. Clear the review queue before releasing.
Blocked runs investigated
Blocked tests could be hiding real issues behind environment or data problems. Investigate and resolve or explicitly accept the risk.
Flaky tests quarantined
Flaky tests should be quarantined so they do not pollute the run result. If flaky tests are still active, they add noise to your readiness signal.
Environment version matches release
Confirm the environment is running the version you intend to release. Running tests against an older version gives false confidence.
What is a release blocker?
A release blocker is any condition that makes the release unreliable:
| Blocker | Why it blocks |
|---|---|
Required/Critical test Verified Failed | Primary workflow is confirmed broken |
Required test Blocked (uninvestigated) | Cannot confirm whether the workflow works |
| Open Critical severity issue | Known significant defect in the release |
Required test To Verify with Needs Review | Ambiguous result on a must-pass test |
When to override
Not every failure is a true blocker. You may proceed when:
- only Low priority tests failed and the failures are well-understood
- a known issue has a documented workaround and does not affect core workflows
- quarantined (flaky) tests are the only non-passing items
- a test is blocked due to a test-data issue, not a product defect
Document the override decision so the team has an audit trail.
Recommended run cadence
| Event | What to run |
|---|---|
| After every deploy | Smoke suite (quick health check) |
| Before every release | Full regression suite |
| After a hotfix | Smoke suite + the specific tests related to the fix |
Define your "Definition of Ready"
The checklist above is the default. To make it your team's policy — and have Certyn apply it automatically — write a Definition of Ready into your project wiki under Testing Rules & Guidelines. Certyn reads the wiki on every conversation, so a release-readiness question gets evaluated against whatever you put there.
Paste this template into the wiki and edit the thresholds to match your team:
## Definition of Ready
A release is **ready** when, for the target environment and the candidate version:
**Hard blockers — do not ship:**
- No open Critical or Major bugs.
- No real (non-flaky) failures on Required/Critical tests.
- Overall pass rate ≥ 95%.
- No unresolved error regression in Sentry for this version.
- The regression suite has run against this version within the last 24 hours.
**Warnings — ship only with sign-off:**
- Known-flaky tests are the only non-passing items.
- Unresolved "Needs Review" items on non-required tests.
- Coverage gaps on areas changed in this release.
**Override:** a blocker may be waived with a documented reason and an owner's sign-off.
If no Definition of Ready is present, Certyn falls back to the default checklist above and tells you it used defaults.
Ask Certyn
You can ask Certyn directly: "Is staging ready for release?" It runs the Release Readiness skill — checking your latest run results, open issues, pass rate, connected trackers, and Sentry, discounting flaky noise — then returns a ranked blocker list and a verdict with evidence, applying your wiki Definition of Ready if you've set one.