For QA teams that keep manual test cases in TestRail, Zephyr Scale, Xray or Azure Test Plans.
Your setup
- Test cases in one of these tools, grouped in suites.
- A web app on a test environment that Certyn can reach, and a test account.
- You want results back in the same tool, on a schedule or after each deploy.
Say this to Certyn
Connect TestRail and run our Regression suite against staging every night at 2am. Write the results back to TestRail.
Or: We connected Xray. Run the Checkout suite in project ABC against staging after each deploy, and write the results back.
Connect the tool
| Tool | Where you connect | What you need |
|---|---|---|
| TestRail | A form in the chat | Base URL, user email and an API key (My Settings → API Keys). API access must be on under Administration → Site Settings → API. |
| Zephyr Scale (Cloud, Server or Data Center) | The Connections page | An API token. The base URL only for Server or Data Center. |
| Xray | The Connections page | A client ID and client secret from an Xray API key pair. |
| Azure Test Plans | The Connections page | The organization URL and a personal access token with Test Plans read and write. |
What Certyn does
- Connects the tool (see the table). For Zephyr, Xray and Azure, connect on the Connections page first, then come back to the chat.
- Finds the suite you name and shows you its cases, so you can check it. In the TestRail form you pick the suites. To fill a TestRail run you already opened, give its number; Certyn uses that run.
- Creates a Certyn suite for it. It asks you first.
- If the app needs a sign-in, opens the sign-in form for the test account.
- Sets the trigger: a time with your time zone, and/or after each deploy. You approve.
- Each run pulls the current cases from the tool, runs them in a real browser, and writes each result back.
What you get
- A run in your tool with a result per case: Passed, Failed, Blocked, or left untested for aborted or skipped cases.
- A Certyn run report, and one finding per real product bug, with evidence.
Limits
- Only the latest attempt per case is written back. A retry does not post twice.
- TestRail cases without a numeric id are skipped when Certyn opens a run.
- Xray cannot list Jira projects. Type the project key (for example
ABC); the suites load after that. - Certyn does not edit cases in your tool. Fix an out-of-date case there.
- After each deploy fires when the environment gets a new version. Your pipeline sets it: see Run a suite after each deploy.
- The CI Runs API and the GitHub Action start only Certyn suites of test cases. They cannot start these suites; use a schedule or after each deploy.