Skip to content
RecipesDoc

Write Tests for What Each Merge Changed

After a merge to main, or each night, Certyn reads what changed, writes tests for it in your repo, and opens a pull request.

For teams that ship features faster than they write tests, and want new tests to follow each change.

Your setup

  • App code and tests in a GitHub repo (Playwright, Cypress or similar).
  • A test environment that gets each merge, for example staging.
  • GitHub Actions, or no CI change at all (nightly).

Say this to Certyn

When something is merged to main in acme/web, write tests for what changed and open a PR.

What Certyn does

  1. Opens the GH_TOKEN form if needed: Contents and Pull requests, Read and write.
  2. Asks which path you want:
    • Each merge (GitHub Actions). Opens the ci_setup form, which creates an API key. You save it as the CERTYN_API_KEY repo secret. Certyn then gives you a workflow on push to main (or opens it as a PR if the token has the workflow permission). It sends an instruction with the commit:
      - uses: certyn-one/action@v1
        with:
          api_key: ${{ secrets.CERTYN_API_KEY }}
          project_slug: my-app
          environment_key: staging
          wait_for_completion: false
          instruction: |
            Commit ${{ github.sha }} was merged to main of acme/web.
            Start a run now: write tests for what this commit changed,
            in our style, run them on staging, and open one PR.
      
      The instruction starts a Certyn chat that acts on it and starts the run.
    • Each night. Creates an agent process with the goal write tests for what was merged to main since yesterday and open one PR, and a nightly schedule in your time zone.
  3. Each run reads the diff, writes tests next to the related ones, runs them, and opens a pull request from a certyn/... branch.

What you get

  • A pull request with tests for each merge, or one per night.
  • A Certyn issue with evidence if the new code is broken.

Limits

  • Certyn GitHub App rules run Certyn suites only, not a goal like this. Use the action or the nightly process.
  • A chat started from CI cannot ask you to approve, so it cannot change settings. Do the setup (token, repo) in the web chat first.
  • A scheduled run has 60 minutes. A busy day can be too much for one run; then use the per-merge path.
  • Each run uses plan time.