# Replay checks outside GitHub

> Run pyrula ci run from a GitLab CI job or any shell, with the same inputs as the GitHub replay-check Action.

Source: https://docs.pyrula.io/getting-started/recipes/ci-outside-github/

The `replay-check` Action is a wrapper around one command, `pyrula ci run`. Anything that can run
that command with a token and a Python environment can run the check: GitLab CI, CircleCI,
Jenkins, a Makefile. What a pass means, the outcomes and the exit codes are on
[Replay checks in CI](https://docs.pyrula.io/workflows/guides/replay-ci.md); this page covers only what changes off
GitHub.

## Inputs

| Action input | `pyrula ci run` |
|---|---|
| `suite` | `--suite NAME` |
| `project` | `--project ID` |
| `environment` | `--environment NAME` (default `production`) |
| `token` | The environment variable `PYRULA_TOKEN`. Never a flag |
| `fixture-store-url` | The environment variable `PYRULA_FIXTURE_STORE_URL` |
| `cloud-url` | `--cloud-url URL` or `PYRULA_CLOUD_URL`, when Cloud is not the hosted service |
| `timeout` | `--timeout S` |
| `idempotency-suffix` | `--idempotency-key KEY` (see [The run key](#the-run-key)) |
| `client-version` | Your own `pip install` pin |
| outputs `verdict`, `json-path`, `junit-path` | `--json PATH`, `--junit PATH`, `--summary PATH`, and the exit code |

Set up the suite, the CI token, the fixture store URL and the `[tool.pyrula.ci]` target map the
same way as for GitHub ([Set up](https://docs.pyrula.io/workflows/guides/replay-ci.md#set-up)). Install your project
first, since the check imports your code, then `pyrula[cloud]`, which brings the `ci` command.

## GitLab CI

This job has not been run on GitLab. The flags in it and in the shell form below are checked
against the command's own parser, and the key and exit-code behaviour is read from its source.

Store the token and the fixture store URL as masked CI/CD variables named `PYRULA_TOKEN` and
`PYRULA_FIXTURE_STORE_URL`, and the project id as `PYRULA_PROJECT`. The job fails on any exit
code other than 0.

```yaml
replay-check:
  image: python:3.13
  rules:
    # Same-project merge requests only: this job runs the change's code with your token.
    - if: $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_SOURCE_PROJECT_PATH == $CI_PROJECT_PATH
  script:
    - pip install -e . "pyrula[cloud]"
    - >
      pyrula ci run
      --suite checkout-regressions
      --project "$PYRULA_PROJECT"
      --idempotency-key "gitlab-$CI_PIPELINE_ID-$CI_JOB_ID"
      --json report.json --junit report.xml --summary report.md
  artifacts:
    when: always
    paths: [report.json, report.md]
    reports:
      junit: report.xml
```

`artifacts:reports:junit` shows each case in the merge request. The JSON report is
`pyrula.replay-ci/1`, as on GitHub ([JSON report](https://docs.pyrula.io/workflows/guides/replay-ci.md#json-report)).

## Any shell

```bash
export PYRULA_TOKEN=...              # from your CI's secret store, never a flag
export PYRULA_FIXTURE_STORE_URL=...
pip install -e . "pyrula[cloud]"
pyrula ci run --suite checkout-regressions --project proj_123 \
  --idempotency-key "$BUILD_ID" \
  --json report.json --junit report.xml --summary report.md
echo "exit code $?"
```

Before the first pipeline, you can see the command's first refusal locally. With no
`[tool.pyrula.ci]` targets configured it exits 2 before any network call:

```bash
status=0
pyrula ci run --suite checkout-regressions --project proj_123 --idempotency-key demo || status=$?
test "$status" -eq 2
```

The exit code is the verdict: 0 every case compatible, 1 a failure, 2 usage or configuration, 3 a
case could not be checked, 4 infrastructure, 5 timeout. Gate on non-zero, and use 3 to 5 to tell a
broken pipeline from a broken change.

## The run key

Cloud holds one suite run per key. On GitHub the command builds it from `GITHUB_RUN_ID`,
`GITHUB_RUN_ATTEMPT` and `GITHUB_JOB`. Off GitHub it reads none of them: without
`--idempotency-key` every invocation is a new suite run, and `--idempotency-suffix` has nothing
to append to.

- Pass a key that is stable for one attempt and different for matrix legs: pipeline id plus job
  id above. A job retried in GitLab has a new job id and starts a new suite run.
- Reuse a key and the command finds the first run: a key already reported exits 3 with the run's
  link. Use this to resume a job that was cut off before it reported.
- A key is at most 128 characters. A suite run counts against the plan's evaluation runs per
  month.

## What GitHub adds that these do not

The command records the repository, commit, ref, pull request number and run URL from GitHub's
`GITHUB_*` variables, and nothing from GitLab's or any other runner's, so a run started elsewhere
carries none of them. The Action also masks the secrets in the log, writes the job summary and
annotates failing cases; in another runner, print `report.md` or publish the JUnit file instead.