Skip to content
RecipesDoc

A Sentry Error Becomes a Reproduced Bug with Evidence

Certyn reads a Sentry error, reproduces it on your test environment, files an issue with steps and screenshots, and can comment back on Sentry.

For teams who see errors in Sentry and need to know which user steps cause them.

Your setup

  • Errors from your web app or API land in Sentry (sentry.io).
  • A staging environment and a test account in Certyn.
  • You want the steps that cause the error, as an issue a developer can follow.

Say this to Certyn

Sentry issue SHOP-1A3 started after yesterday's release. Find out what users do to hit it, reproduce it on staging, and file a bug.

What Certyn does

  1. If Sentry is not connected, opens the Sentry connect form: an auth token (scopes project:read, event:read, org:read), your org slug and your project slug.
  2. Reads the Sentry issue and its latest event (sentry.issue.get): the exception, stack trace, tags, breadcrumbs and release. Without an id, it lists recent unresolved issues (sentry.issue.list) so you can pick one.
  3. Searches for a Certyn issue that already matches (app.find).
  4. Turns the breadcrumbs and the failing URL or endpoint into steps. It tries a simple flow on the chat's Machine while you watch, or says the run length and starts an explore run with the steps, the error text and the Sentry link in the goal (app.run.create).
  5. Records a verdict: reproduced, partly reproduced, not reproduced, or blocked.
  6. If it reproduces, files one Certyn issue with steps, expected and actual result, screenshots, and the Sentry link.
  7. If you agree, posts a comment on the Sentry issue with the steps and the Certyn issue link (sentry.comment.create).

What you get

  • Steps that cause the error, with evidence either way.
  • One Certyn issue per real bug, linked to the Sentry issue in its text.
  • Optional: on the Connections page, add Sentry's webhook. New Sentry issues (or only those that match your alert rules) then appear in Certyn as issues to review, with the release.

Limits

  • Only the chat reads Sentry. A run gets the Sentry details as text in its goal.
  • A comment needs a token that can write to issues (Sentry scope event:write). With only the read scopes, the comment fails; ask Certyn to show the text so you can paste it.
  • An error that needs real customer data or production traffic may not reproduce on staging. Describe the data in the request, or add similar data on staging.
  • The forms connect sentry.io. There is no field for a self-hosted Sentry URL.