Work in Your Repo

You already have tests: Playwright, Cypress, Selenium, pytest, Postman, or something else. You do not need to import or rewrite them. Give Certyn access to your repo, and it works with them like a QA engineer who joins your team:

• Run your suite the way your CI runs it, against the environment you pick, and tell you what failed and why.
• Write new tests in your folders, with your helpers and naming.
• Fix a broken or flaky test and open a pull request.

1. Give Certyn a GitHub token

In Ask Certyn, say run my tests from github.com/acme/web. Certyn opens a form for a GH_TOKEN and links the GitHub page where you create it. Make a fine-grained token:

• Repository access: only the repo with your tests.
• Permissions: Contents and Pull requests, both Read and write. Contents: Read alone is enough if Certyn should only run your tests.

The token is saved as a secret project variable. Certyn never shows it again, not to you and not in the chat. You can also add it yourself in Project settings → Variables.

GitLab, Bitbucket and other hosts work the same way: save their token as a secret project variable (for example GITLAB_TOKEN) and tell Certyn which one it is.

2. Pick where your tests run

A cloud Machine starts clean every time. On your runner, Certyn can install anything, and a workspace folder keeps your clone, tools and caches between runs, so the next run starts fast.

3. Run your suite

Ask in the chat: run the Playwright tests in acme/web against staging. When a Machine is open, Certyn first tries one test there to check the setup. Then it starts a run, which:

1. Clones your repo and reads your CI file to find how you install and run the tests.
2. Points the tests at the environment you picked, with its URL and variables.
3. Runs the whole suite, then runs each failed test once more.
4. Decides why each test failed: a product bug, a broken test (out of date), a flaky test, or a setup problem.

The result comes back like any other run:

• One run report with the totals and the failures grouped by cause.
• One issue for each product bug, with the failing tests as evidence. Ten tests that fail on one broken page are one issue, not ten. Broken and flaky tests are listed in the report, not filed as issues.

A run takes up to 60 minutes by default. For a long suite, Certyn asks for more time, up to 4 hours. A run uses plan time for as long as it runs.

To run it every night or after each deploy, ask Certyn to schedule it. It saves the run as a process and sets its schedule. A scheduled run has 60 minutes.

4. Fix a test or add one

Ask: fix the failing checkout test and open a PR, or add a test for the coupon field. Certyn:

1. Creates a branch named certyn/... from your default branch.
2. Makes the smallest change, in your style. It never makes a test pass by weakening it. A test that fails because the app is wrong is a bug, so Certyn reports it instead.
3. Runs the test again, several times for a flaky one.
4. Opens a pull request that says what failed, why, what changed, and how it was checked.

Safety

• Certyn works only on its own certyn/... branches. It never pushes to your default branch, force-pushes, merges, or closes pull requests.
• Keep branch protection and required reviews on your default branch. They decide what gets merged, not Certyn.
• In a scheduled or CI run, Certyn opens a pull request only when the run's goal asks for one.
• Your tests run against the environment you pick. Use a test environment and a test account, as for any run.

Related