# Record once, iterate free

> Keep the last pyrula dev run as a case, rerun it offline with a new prompt or model while its tool results replay from the recording, and check saved cases in CI with pytest. No Cloud account.

Source: https://docs.pyrula.io/workflows/guides/record-once-iterate-free/

`pyrula dev` keeps the newest finished run of its target as a case file in `.pyrula/last/`: inputs, journal and every recorded step, model and tool result. `pyrula case run` reruns it against your current code, answering matching calls from the recording, so a new system prompt or model can be tried without calling a tool again. A write the recording cannot answer is refused. Copied into your repository, the same file becomes a pytest test. None of this needs a Cloud account or a network.

This page uses Python 3.11 to 3.15 and a POSIX shell, in an empty directory.

## 1. The agent

```bash
pip install "pyrula[agents]"
```

Save this as `app.py`. It is the agent `pyrula init` writes: one tool and a scripted model, so it runs with no key. `TOOL_CALLS` counts the times the tool really runs.

```python
import asyncio
from typing import Any

from pyrula.agents import AgentContext, ToolContext, agent, tool
from pyrula.agents.testing import FakeLLM, FakeResponse
from pyrula.agents.testing.runner import AgentRunner

TOOL_CALLS: list[str] = []


@tool(read_only=True)
async def lookup_order(ctx: ToolContext, order_id: str) -> dict[str, Any]:
    """Look up an order by id."""
    TOOL_CALLS.append(order_id)
    return {"status": "shipped", "total_cents": 4200}


exchange = [
    FakeResponse(
        tool_calls=[{"name": "lookup_order", "id": "t1", "input": {"order_id": "A-1042"}}]
    ),
    FakeResponse(text="Order A-1042 has shipped. Total $42.00."),
]
llm: Any = FakeLLM(exchange * 5)


@agent(llm=llm, timeout=60)
async def support(ctx: AgentContext, message: str = "Where is order A-1042?") -> str:
    return await ctx.llm.run([{"role": "user", "content": message}], tools=[lookup_order])


if __name__ == "__main__":
    asyncio.run(AgentRunner(support, llm=llm).run(message="Where is order A-1042?"))
    print("tool calls:", TOOL_CALLS)
```

## 2. Record a run under `pyrula dev`

```bash
pyrula dev app.py
```

`dev` runs the agent once at startup. When a run finishes it writes `.pyrula/last/support.case` and prints where; the newest finished run replaces the file. The dev UI shows the `pyrula case run` command for each run. Pass `--no-save-last` to keep nothing.

The commands from here on run in a second terminal. In a script, start `dev` in the
background and wait for the file:

```bash
pyrula dev app.py --no-ui > dev.log 2>&1 &
for _ in $(seq 150); do [ -f .pyrula/last/support.case ] && break; sleep 0.2; done
kill $!
ls .pyrula/last
```

## 3. Rerun it with a new prompt

Write the system prompt to try into a file and rerun the case with it:

```bash
printf 'Answer in one short sentence.\n' > prompt.txt
pyrula case run .pyrula/last/support.case --target app.py:support --from-start \
  --prompt-file prompt.txt
```

```text
{"key": "llm#0", "kind": "llm", "provenance": "live", "type": "call"}
{"key": "t1", "kind": "tool", "provenance": "reused", "type": "call"}
{"key": "llm#1", "kind": "llm", "provenance": "live", "type": "call"}
{"changed_calls": [], ..., "live_calls": 2, "provenance": {"changed": 0, "live": 2, "reused": 1, "simulated": 0}, ..., "status": "complete", "type": "outcome", ...}
```

`--from-start` reruns the whole recorded run. With `--prompt-file` (or `--model ID`), every model call runs live on the agent's own client, because a recorded answer was made under the old prompt. These are real provider calls and are billed. The tool call asks for the same order, so its recorded result answers it (`reused`). A tool call whose input changed runs live only when the tool is declared `read_only=True`; any other tool is a write, and the run stops with `write_not_allowed` unless you pass `--allow-write tool:NAME`.

Other flags: `--llm-ref MODULE:ATTR` runs on another client, `--replay-only` answers every call from the recording and stops at the first it cannot, and `--live-budget N` or `--max-cost-usd X` bound what runs live.

## 4. Keep it as a test

Copy the case into the repository and run it with pytest:

```bash
mkdir -p cases
cp .pyrula/last/support.case cases/
pytest -q --pyrula-cases cases
```

`--pyrula-cases DIR` collects every `.case` file under `DIR` as a test. Each replays the case's recorded journal against the code registered under its target name and fails on the first difference, such as a step in another order or a recorded call the code no longer makes. Nothing runs live. Targets are imported from `app.py` and `agents.py` in the current directory; name other modules with `--pyrula-app FILE`.

For a hand-written test, the `case_fixture` fixture loads a case file, and
`assert_case_compatible` runs the same check:

```python
from pyrula.workflows.testing.cases import assert_case_compatible


def test_support_case(case_fixture):
    case = case_fixture("cases/support.case")
    assert case.manifest.target_name == "support"
    assert_case_compatible(case)
```

In CI, the same `pytest --pyrula-cases cases` line needs no model key, store or Cloud account. Without `--pyrula-cases` the plugin collects nothing.

## Case file contents

A case file holds inputs, every journal event and every recorded model and tool result, as zstd-compressed JSON, written owner-only (mode 0600) in `.pyrula/`, which `pyrula init` adds to `.gitignore`. Read a case before you copy it into a repository: a recorded prompt or tool result can carry customer data or a secret. A target with capture level `none` (`capture="none"` or `PYRULA_CLOUD_CAPTURE=none`) is never saved. A failed save is reported on stderr and `pyrula dev` tries again after the next run.