Skip to content
RecipesDoc

Which Run to Start, How Long It Takes and What It Costs

The run types you can ask for in the chat, their time limits, and how they use plan time.

For anyone who wants to know which run to ask for, and how much plan time it uses.

Your setup

  • A web app or API on one or more environments.
  • Some saved test cases and issues.
  • A plan with monthly plan time.

Say this to Certyn

I have 20 minutes before the release. What should you run to check checkout on staging, and how much plan time will it use?

What Certyn does

  1. Picks the run type from what you ask, says the type and length, and starts it only after you agree (app.run.create).
You wantRun typeTime limit
One check, from a sentencequick60 minutes, fixed
An area, a feature, or your team's own suite from the repoexplore with a goal and minutes10 to 240, default 60
Saved test casestest_case, up to 50 at once. A whole suite: app.process.run5 minutes each
An API from its OpenAPI specapi with spec_url18 minutes
Check that a fixed bug is goneretest with the issue5 minutes
A failed, blocked or aborted run againretry with that runSame as before
  1. quick and explore are the same kind of run; explore lets you set the length and a name.
  2. The limit is a safety stop. Most free-text runs end well before it. For the example above: your checkout test cases, or a short explore run.

What you get

  • Plan time is the time a run really takes, not its limit, times the testing depth: Standard 1×, Deep 3×, Expert 5×.
  • Testing depth is set in Project settings, on the Testing depth tab, for each kind of task: Test cases, Focused checks, Broad exploration, Security testing. Deep uses a stronger model to plan and judge; Expert uses it for every step. The default is Standard.
  • A free-text run that plans a broad sweep moves up to your Broad exploration depth.
  • Chat answers use plan time by model cost: 1 cent is 1 plan minute. The depth picker under the chat box is for chat answers only.
  • The chat's Machine uses plan time while it runs, as a run does.

Limits

  • You cannot set the depth of one run in the chat. Change the Testing depth tab first.
  • retest needs an issue linked to a test case or a tracker issue.
  • Scheduled and deploy-triggered runs of an agent process have 60 minutes.