GitLab and Other CI

Use the CI Runs API for GitLab CI, Jenkins, CircleCI, Azure DevOps, Bitbucket Pipelines, Buildkite, or any runner that can execute shell commands.

Only want to send results in?

This page covers asking Certyn to run a suite and gating on it. If instead you want your existing Playwright or JUnit suite to report into Certyn, that is a single curl and no polling. See Bring Your Test Framework.

The gate has three steps:

Create a CI run

POST to /api/ci/runs with projectSlug, environmentKey, and either processSlug or tags.

Poll status

GET /api/ci/runs/{runId} until the run reaches completed or cancelled.

Fail the job when needed

Exit non-zero when failed > 0 or blocked > 0.

GitLab CI

Store CERTYN_API_KEY as a protected GitLab CI/CD variable.

stages:
  - test

certyn-smoke:
  stage: test
  script:
    - |
      set -euo pipefail
      body=$(jq -nc \
        --arg project "$CERTYN_PROJECT_SLUG" \
        --arg environment "$CERTYN_ENVIRONMENT_KEY" \
        --arg repository "$CI_PROJECT_PATH" \
        --arg ref "$CI_COMMIT_REF_NAME" \
        --arg sha "$CI_COMMIT_SHA" \
        --arg url "$CI_PIPELINE_URL" \
        '{projectSlug:$project,environmentKey:$environment,processSlug:"smoke-suite",repository:$repository,ref:$ref,commitSha:$sha,event:"pipeline",externalUrl:$url}')

      response=$(curl -sS -X POST "$CERTYN_API_URL/api/ci/runs" \
        -H "Content-Type: application/json" \
        -H "X-API-Key: $CERTYN_API_KEY" \
        -H "Idempotency-Key: $CI_PIPELINE_ID-$CI_JOB_ID" \
        -d "$body")
      run_id=$(echo "$response" | jq -r '.runId')

      for i in $(seq 1 60); do
        status=$(curl -sS -H "X-API-Key: $CERTYN_API_KEY" "$CERTYN_API_URL/api/ci/runs/$run_id")
        state=$(echo "$status" | jq -r '.state')
        failed=$(echo "$status" | jq -r '.failed')
        blocked=$(echo "$status" | jq -r '.blocked')
        interval=$(echo "$status" | jq -r '.retryAfterSeconds // 15')
        app_url=$(echo "$status" | jq -r '.appUrl // empty')
        echo "state=$state failed=$failed blocked=$blocked appUrl=$app_url"

        if [ "$state" = "completed" ] || [ "$state" = "cancelled" ]; then
          [ "$failed" = "0" ] && [ "$blocked" = "0" ]
          exit $?
        fi

        sleep "$interval"
      done

      echo "Timed out waiting for Certyn run"
      exit 1
  variables:
    CERTYN_API_URL: https://api.certyn.io
    CERTYN_PROJECT_SLUG: my-app
    CERTYN_ENVIRONMENT_KEY: staging
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

Generic Shell Step

This example works for most CI systems with curl and jq available.

set -euo pipefail

CERTYN_API_URL="${CERTYN_API_URL:-https://api.certyn.io}"
CERTYN_PROJECT_SLUG="${CERTYN_PROJECT_SLUG:-my-app}"
CERTYN_ENVIRONMENT_KEY="${CERTYN_ENVIRONMENT_KEY:-staging}"

body=$(jq -nc \
  --arg project "$CERTYN_PROJECT_SLUG" \
  --arg environment "$CERTYN_ENVIRONMENT_KEY" \
  '{projectSlug:$project,environmentKey:$environment,processSlug:"smoke-suite",event:"pipeline"}')

response=$(curl -sS -X POST "$CERTYN_API_URL/api/ci/runs" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $CERTYN_API_KEY" \
  -H "Idempotency-Key: $CI_JOB_ID" \
  -d "$body")
run_id=$(echo "$response" | jq -r '.runId')

for i in $(seq 1 60); do
  status=$(curl -sS -H "X-API-Key: $CERTYN_API_KEY" "$CERTYN_API_URL/api/ci/runs/$run_id")
  state=$(echo "$status" | jq -r '.state')
  failed=$(echo "$status" | jq -r '.failed')
  blocked=$(echo "$status" | jq -r '.blocked')
  interval=$(echo "$status" | jq -r '.retryAfterSeconds // 15')

  if [ "$state" = "completed" ] || [ "$state" = "cancelled" ]; then
    [ "$failed" = "0" ] && [ "$blocked" = "0" ]
    exit $?
  fi

  sleep "$interval"
done

exit 1

Notes

Use Idempotency-Key so CI retries do not create duplicate Certyn runs.
Use processSlug for a known suite such as smoke-suite.
Use tags instead of processSlug when a pipeline should select tests dynamically.
See CI Runs API for the full request and status schema.