Skip to content
RecipesDoc

Triage Findings in Review, Confirm or Close Each with a Reason

Go through the findings waiting for review with Certyn, confirm the real bugs, and close the rest with a reason that stops the same finding from coming back.

For QA leads and developers who review what Certyn's runs found before it becomes tracked work.

Your setup

  • Certyn runs have found possible bugs. They wait in review on the Findings page.
  • Some are real, some are expected behavior, some are duplicates.
  • You do not want to see the same dismissed finding again next week.

Say this to Certyn

Go through the findings in review with me. Recommend confirm or close for each, most severe first.

What Certyn does

  1. Lists the findings in review, most severe first (app.query on issues with status in_review).
  2. For each one, reads the issue and its evidence (app.issue.get): steps, screenshots, the run. If the evidence is weak, it can try the steps again on the chat's Machine.
  3. Recommends confirm or close with a reason, and asks you before each change.
  4. Applies your choice (app.issue.update): confirmed, or a close reason with a short note. For duplicate, the note names the other issue.
  5. A confirmed finding becomes a tracked Certyn issue. It goes to Jira, Linear or GitHub only if you ask; Certyn asks before it files there.

Which close reason to pick:

ReasonUse it whenSame finding later
not_a_bugIt is expected behaviorNot filed again
duplicateAnother issue already tracks itNot filed again
wont_fixReal, but you will not fix itNot filed again
fixedIt is fixedFiled again (a regression)
cannot_reproduceNobody can make it happenFiled again
otherNone of the aboveFiled again

What you get

  • A clean review queue: real bugs confirmed, the rest closed with a reason and a note.
  • Fewer repeat findings. When a later finding closely matches an issue closed as not_a_bug, duplicate or wont_fix, no new issue is created, and the run is told about the earlier decision.

Limits

  • The match is by similarity, not exact text. A finding that is clearly different (another page, cause or condition) is still filed, on purpose. A repeat written very differently can slip through; close it again with the same reason.
  • Issues closed before close reasons existed do not stop repeats.
  • Use fixed only for real fixes. It does not stop the bug coming back, which is what you want.
  • You can also do this by hand: on the Findings page, confirm a finding or close it and pick the reason.