For teams whose app is only on a VPN or internal network, or whose tests need tools that take long to install (a JDK, the .NET SDK, k6). A self-hosted runner puts Certyn's Machine on your own host.
Your setup
- Tests in a repo: Selenium with Java and Maven, Selenium with C# and NUnit, a k6 script, or any other suite.
- A Linux host with Docker that can reach the app. The runner needs only outbound HTTPS.
- Your current CI (for example Jenkins) stays as it is.
Say this to Certyn
Our Selenium Java tests (Maven) run in Jenkins. The app is only on our VPN at https://staging.acme.internal. Run them and tell me why they fail.
Other examples:
- Run our NUnit Selenium tests from github.com/acme/e2e on our runner against staging.
- Run our k6 script loadtests/checkout.js on our runner against staging and tell me which thresholds failed.
What Certyn does
-
If you have no runner yet, opens the runner form. It shows one
docker runcommand. Before you run it on your host, add this line so tools and caches stay between runs:-e WORKSPACE_PATH=/srv/certyn-workspace \The form waits until the runner is online, then sends the default environment's runs to it.
-
Opens a form for repo access:
GH_TOKENfor GitHub, or your host's token (for exampleBITBUCKET_TOKEN) as a secret project variable. -
Starts a Machine on the runner and checks the app URL from inside your network.
-
Installs what the suite needs into
/workspaceonce (see the table). Package caches go there without extra setup: npm, Playwright browsers, pip, Maven (/workspace/.m2), Gradle and NuGet (/workspace/.nuget). -
Reads your CI file (
Jenkinsfile, workflow) or build files for the test command, passes the environment URL ($TARGET_BASE_URL) to the setting your tests read, and runs one test. -
Runs the whole suite as a background run, and reads its report to decide the cause of each failure.
Run commands
| Framework | Installed once into /workspace | Typical command |
|---|---|---|
| Selenium (Java, Maven) | A JDK and Maven | mvn test (Certyn reads the Surefire XML) |
| Selenium (C#, NUnit) | The .NET SDK version from global.json or the project files | dotnet test --logger "junit;LogFilePath=junit.xml" (or a TRX logger) |
| k6 | The k6 binary | k6 run -e BASE_URL=$TARGET_BASE_URL loadtests/checkout.js |
| Playwright, Cypress, pytest, others | What a cloud Machine has, plus your packages | Your CI's command |
For C#, the URL goes in as a .runsettings parameter or an environment variable, whichever your tests read. Chromium and chromedriver are on the Machine.
For k6, k6 exits with an error when a threshold fails. Certyn reads the k6 summary and reports each threshold, passed or failed, with the measured value, plus failed checks and error rates.
What you get
- A run report with failures grouped by cause: product bug, broken test, flaky test, setup problem.
- One finding per real product bug, with evidence.
- Fast later runs: the tools, packages and clone are already in
/workspace. - Your CI unchanged. To keep its history in Certyn too, push its JUnit or Surefire XML: see Push JUnit results.
- If you ask: the same run on a schedule or after each deploy.
Machine size for k6 and heavy suites
By default a runner gives each Machine 1 CPU core and 2.5 GB of memory. Raise it on the runner command, up to 16 each:
-e CONTAINER_CPU_CORES=4 -e CONTAINER_MEMORY_GB=8 \
Certyn is not a load-testing service. The load comes from one Machine: one container, one host, one IP address. That is enough for a baseline or a small load test. For high or distributed load, use a load-testing service. Put load on staging, not production.
Limits
- The runner form routes only the project's default environment. For another environment, set Execution target to Self-hosted runner on that environment in Settings → Environments.
- Without
WORKSPACE_PATH, every tool and dependency installs again at every run. - All runs on this runner share the workspace folder. Give each runner pool its own folder, and use it only for your team.
- Machines run Linux. Tests that need Windows (Windows-only APIs or a Windows browser) do not run. Another browser must be installed on the runner first.
- One shell command can run 30 minutes; longer jobs run in the background. A run has 60 minutes by default, up to 4 hours.
- On Certyn's cloud these suites work too, but the JDK or SDK installs again at every run, which adds minutes, and you cannot change the Machine's size.