What-if runs

A what-if forks a finished run at a point, changes something after it, and has one of your workers run the rest as a new linked run. Each call after the fork is labelled reused, live, simulated or changed. Anything that could touch the outside world is stopped unless you allow it.

It uses the same engine as saved cases but needs no fixture store and creates no case. The entry point is the Cloud REST API. To try a change offline, see Running a case.

Requirements

• A Cloud project with a worker in the run’s environment, on a pyrula version that supports what-ifs. The worker must still hold the source run (Limits).
• A finished run (completed, failed, cancelled or abandoned). A run in progress answers 409.
• A full-scope user token and the project member role. read and ci tokens and platform staff cannot start one.

export BASE="https://YOUR_API_HOST/api/v1/projects/YOUR_PROJECT_ID"
export PYRULA_API_TOKEN=...   # a full-scope token

Start one

A what-if forks after a run_seq; calls up to it are replayed from the recording. To rerun a failed step, fork at the event just before its step:pending. List the step events and read clientSeq:

curl -s -H "Authorization: Bearer $PYRULA_API_TOKEN" \
  "$BASE/runs/fulfill_order/run_abc123/events?environment=production&kind=step:pending&kind=step:error"

If the failing step:pending has clientSeq 12, fork after 11. clientRequestId is a UUID per attempt; resending it returns the existing what-if (200).

{
  "source": {"runId": "run_abc123", "agent": "fulfill_order", "forkAfterSeq": 11},
  "overrides": {"inputsPatch": {"coupon": null, "shipping": {"express": false}}},
  "label": "Without the coupon, standard shipping",
  "clientRequestId": "6f1d0b9e-5a3c-4b7e-9d21-0c8e4a7f3b52"
}

curl -s -X POST -H "Authorization: Bearer $PYRULA_API_TOKEN" -H "Content-Type: application/json" \
  -d @whatif.json "$BASE/whatifs?environment=production"

Request	Does
POST $BASE/whatifs/preflight	Same body, starts nothing. Reports the worker, events reused, budget and warnings.
GET $BASE/whatifs/{id}	state: queued, running, complete, failed, not_run or cancelled. not_run carries notRunReason (Error codes).
POST $BASE/whatifs/{id}/cancel	Stops one in flight.
GET $BASE/runs/{agent}/{runId}/whatifs	Lists what-ifs started from a run.
GET $BASE/runs/{targetName}/{runId}/events	The what-if’s own events (names from the detail read).

A what-if counts against your plan’s monthly and concurrent comparison limits (402 and 409), and its run against runs per month.

Overrides

All edits go in overrides; omit a field to change nothing. The object is capped at 64 KiB and 16 levels of nesting.

Inputs

inputsPatch is an RFC 7396 merge patch over the recorded inputs: a null deletes a key, objects merge, anything else replaces.

from pyrula.cases.policy import apply_merge_patch

recorded = {"order": "A-1", "coupon": "SPRING", "shipping": {"express": True, "country": "DE"}}
patch = {"coupon": None, "shipping": {"express": False}}
assert apply_merge_patch(recorded, patch) == {
    "order": "A-1",
    "shipping": {"express": False, "country": "DE"},
}

The worker applies the patch to its own copy of the inputs, since Cloud may hold only scrubbed ones. A patch containing a redaction placeholder ([REDACTED], [REDACTED]#1, or [redacted: mask failed]) is refused; send a new whole value instead. A patch holds at most 200 object keys and 64 KiB.

If the patch changes the input of a step that ran before the fork, the worker runs from the start (effectiveForkAfterSeq is 0). Unchanged calls still reuse their recordings.

Step results

replace swaps a recorded result. Keys are <kind>:<key>, such as step:charge#0 or tool:toolu_01; the value is the whole new result:

{"replace": {"step:lookup_price#0": {"price": 12.5}}}

Only calls after the fork can be replaced. A replaced result is labelled simulated with replaced: true; later calls whose inputs depend on it run live within the budget. A result of the wrong shape stops the run with missing_context.

rerunLive takes the same keys and sends those calls live despite a matching recording. suppliedResults (calls, tools) works as in a case run.

Model and system prompt

{"model": "claude-sonnet-5", "systemPrompt": "Answer in one sentence."}

Agent runs only, and only when the worker’s runtime can switch models and prompts. Every model call of the top-level agent at or after the fork runs live; sub-agents keep their model. A model with no price is refused with unpriced_model while a cost budget is set.

Budgets and writes

• Writes are denied by default. Any tool or step not declared read_only or effect="read" is a write. A write with no recorded or supplied answer stops the run with write_not_allowed; run:error carries the call (stop_key) and the token that would allow it (stop_token, such as tool:send_email).
• Default budgets are 50 live calls and $1.00. Anyone who can start a what-if can lower them. Raising or clearing a budget, or allowing a write, needs a project admin.
• Cost needs a price. Cloud sends its built-in table (the same as pyrula’s default prices); an operator or project rate wins. A model call with no price stops with live_budget_exceeded. An admin may set "maxCostUsd": null to run on the call budget alone.

{
  "overrides": {
    "livePolicy": {"allowWrites": ["tool:send_email"], "maxLiveCalls": 20, "maxCostUsd": 0.25}
  }
}

An allowed write runs for real, at-least-once. A side effect inside a step declared a read or pure still happens.

Labels

Every call after the fork carries a provenance label on its step:done event:

Label	The call
reused	Returned its recorded result; the inputs matched.
simulated	Was answered by a result you supplied, or by a replace (replaced: true).
live	Ran for real with no recording to compare, or was forced live by rerunLive or a model override.
changed	Ran for real because its inputs differed from the recording.

A call stopped by the policy has no label; the run ends with the stop code instead.

Save as a case

POST $BASE/whatifs/{id}/save-case with {"name": "Order A-1 without the coupon"} saves the what-if’s own run as a case holding the edited inputs and the replaced and live results. It then replays in a replay_only CI suite. Pass criteria to give it checks; a case with none cannot fail a suite. Saving needs a worker with a fixture store that still holds the run.

Limits

• The source run must still be on the worker. On a plain Valkey store a completed run is deleted after completed_ttl (a day by default); with a Postgres archive the worker reads it back (retention). Once gone, the what-if ends whatif_source_missing. Save runs you care about as cases.
• It runs your deployed code. If that code no longer reaches the fork as recorded, the what-if ends not_run (prefix_diverged or version_skew).
• A what-if is deleted with its run when retention removes it. A saved case survives.
• Live calls cost money, and live reads touch real services.

Error codes

A what-if is refused at start (an HTTP error) or ends with one of these reasons, shown as notRunReason or as code on run:error.

Code	When	Do
whatif_source_missing	The worker no longer holds the source run.	Configure the Postgres archive, or save runs as cases.
whatif_source_changed	The source run rebuilt to a different bundle than the what-if started from.	Start a new what-if.
invalid_overrides	An override fails validation: a replaced call before the fork, a bad key, a redaction placeholder, a model on a client that cannot switch, or over the limits.	Fix the override.
unpriced_model	The model has no price while a cost budget is set.	Choose a priced model, or ask an admin to clear maxCostUsd.
write_not_allowed	A write has no allow token and no supplied result.	Supply a result, or ask an admin to allow the write.
live_budget_exceeded	The live-call or cost budget is spent, or spend cannot be observed.	Narrow the edit, or ask an admin to raise the budget.
missing_context	A call has no recording, supply or live path, or a supplied value has the wrong shape.	Supply a valid result.
prefix_diverged	The code no longer takes the recorded path before the fork.	Restore the code, or fork at 0.
version_skew	The worker’s code version cannot replay the recorded steps.	Deploy compatible code, or fork at 0.
capture_incomplete	A call was in flight at the fork, or the journal lacks what the fork needs.	Fork at another point.
no_worker, runtime_too_old	No worker in the environment can run what-ifs, or none is new enough.	Start or upgrade one.
binding_mismatch	The worker’s run store belongs to another project or environment.	Check the worker’s configuration.
runs_quota_exceeded	The monthly run quota is used up.	Check the plan.
runtime_lost, runtime_error	The worker stopped acknowledging, or reported an error.	Check the worker, then start again.
cancelled	The what-if was cancelled.	Start a new one.
no_fixture_store	Saving as a case, and no worker has a fixture store.	Configure PYRULA_FIXTURE_STORE.

HTTP refusals: 402 comparison quota; 403 a budget or write needing an admin, or platform staff; 404 unknown run or project; 409 run unfinished, too many what-ifs running, case not ready or already saved; 422 validation failure.