Skip to content

Release Readiness Playbook

A practical QA checklist for deciding whether an environment is ready to ship.

Release readiness is a risk decision. The goal is not to prove there are no bugs. The goal is to decide whether the remaining known and unknown risk is acceptable.

Default verdicts

Use three verdicts:

VerdictMeaning
ReadyRequired evidence is current and no blockers are known
Ready with warningsNo hard blockers, but known risks need sign-off
BlockedOne or more conditions make release unsafe or unverifiable

Hard blockers

Treat these as blockers unless the project wiki explicitly defines a different policy:

  • required or critical test failed and is not confirmed flaky
  • critical or high severity issue is open for the target release
  • latest required suite did not complete
  • release candidate version was not tested
  • required test is blocked and the cause is unknown
  • production error or security regression is unresolved
  • test evidence is too stale for the release decision

Warning signals

Warnings do not always block release, but they should be visible:

  • pass rate is declining
  • flaky rate is rising
  • non-critical failures are clustered in one area
  • "Needs Review" backlog is growing
  • changed area has weak coverage
  • tests passed only in a different environment or older version
  • external tracker or Sentry data is unavailable

Evidence checklist

Before approving a release, collect:

  • target environment and release version
  • latest smoke and regression run IDs
  • pass rate and trend
  • open critical/high issues
  • flaky tests and quarantine status
  • blocked or needs-review items
  • recent changed areas and matching coverage
  • connected tracker or Sentry signals when configured
  • documented overrides with owner and reason

Certyn readiness flow

  1. Confirm the target environment and version.
  2. Load the project's release-readiness skill if one exists.
  3. Read the project wiki for team-specific thresholds.
  4. Check latest runs, process status, metrics, issues, flakiness, and connected tools.
  5. Separate facts from recommendations.
  6. Return a verdict, blockers, warnings, evidence, and next actions.

Override policy

An override is acceptable only when it is explicit.

Record:

  • what is being waived
  • why the risk is acceptable
  • who owns the decision
  • what mitigation or follow-up exists
  • when the waiver expires

Avoid silent overrides. If a rule is routinely waived, update the project wiki so the rule matches reality.

Good Ask Certyn prompts

Is staging ready to ship? Use the release-readiness skill and include blockers, warnings, evidence, and next actions.
What coverage gaps should block this checkout release?
Summarize the release risk for production based on the latest regression run and open high-severity issues.