For anyone who wants to know which run to ask for, and how much plan time it uses.
Your setup
- A web app or API on one or more environments.
- Some saved test cases and issues.
- A plan with monthly plan time.
Say this to Certyn
I have 20 minutes before the release. What should you run to check checkout on staging, and how much plan time will it use?
What Certyn does
- Picks the run type from what you ask, says the type and length, and starts it only after you agree (
app.run.create).
| You want | Run type | Time limit |
|---|---|---|
| One check, from a sentence | quick | 60 minutes, fixed |
| An area, a feature, or your team's own suite from the repo | explore with a goal and minutes | 10 to 240, default 60 |
| Saved test cases | test_case, up to 50 at once. A whole suite: app.process.run | 5 minutes each |
| An API from its OpenAPI spec | api with spec_url | 18 minutes |
| Check that a fixed bug is gone | retest with the issue | 5 minutes |
| A failed, blocked or aborted run again | retry with that run | Same as before |
quickandexploreare the same kind of run;explorelets you set the length and a name.- The limit is a safety stop. Most free-text runs end well before it. For the example above: your checkout test cases, or a short explore run.
What you get
- Plan time is the time a run really takes, not its limit, times the testing depth: Standard 1×, Deep 3×, Expert 5×.
- Testing depth is set in Project settings, on the Testing depth tab, for each kind of task: Test cases, Focused checks, Broad exploration, Security testing. Deep uses a stronger model to plan and judge; Expert uses it for every step. The default is Standard.
- A free-text run that plans a broad sweep moves up to your Broad exploration depth.
- Chat answers use plan time by model cost: 1 cent is 1 plan minute. The depth picker under the chat box is for chat answers only.
- The chat's Machine uses plan time while it runs, as a run does.
Limits
- You cannot set the depth of one run in the chat. Change the Testing depth tab first.
retestneeds an issue linked to a test case or a tracker issue.- Scheduled and deploy-triggered runs of an agent process have 60 minutes.