Skip to content
certyn logoCertyn
Start Free
RecipesDoc

Playwright Suite in GitHub, Public App

Certyn runs your Playwright tests from GitHub against your environment and reports the failures grouped by cause.

For teams with Playwright tests in a GitHub repo and an app on the public internet, who want one run now with a clear report.

Your setup

  • Playwright tests in a GitHub repo.
  • The app is on the internet, for example a staging URL.
  • Your CI stays as it is.
  • You want to know what failed and why.

Say this to Certyn

Run the Playwright tests in github.com/acme/web against staging and tell me what failed and why.

What Certyn does

  1. Opens a form for a GitHub token, saved as the secret project variable GH_TOKEN. Make a fine-grained token for that one repo with Contents: Read. Add Pull requests: Read and write only for fixes as pull requests.
  2. Saves the repo and test command as project variables (for example GITHUB_REPO) for later runs.
  3. If a Machine is open, clones the repo, installs it the way your CI file does, and runs one test. The environment's URL is $TARGET_BASE_URL on the Machine; Certyn passes it to the variable your playwright.config reads.
  4. Starts a run (app.run.create type explore) with the goal: clone, install, run npx playwright test against this environment, report. A run has 60 minutes by default; for a long suite Certyn asks for more, up to 4 hours.

What you get

  • One run report: the totals, and the failures grouped by cause: product bug, broken test, flaky test, setup problem.
  • One Certyn issue per real product bug, with the failing tests as evidence. A known bug is not filed twice, and it is retested after a fix.
  • Broken and flaky tests in the report, not as issues.
  • If you ask: a fix for a broken test as a pull request from a certyn/... branch. Certyn never merges.

Limits

  • A cloud Machine starts clean, so it installs your dependencies at every run. To keep them, use a self-hosted runner with a workspace folder.
  • A run uses plan time for as long as it runs.
  • Tests that sign in need a test account, saved through the sign-in form, never typed in the chat.
  • An app that is not on the internet needs a runner inside your network.
PreviousRecipesNextRun Your Playwright Suite Every Night