Skip to content
Pyrula
Pre-release: the packages are not on PyPI yet, so pip install does not resolve. Build from source to try it today.

Cancel and terminate

CancelTerminate
Ends the runAt its next step boundary, by the run’s workerAt once, by the store
Needs a workerYes, for this workflow and versionNo
Running stepFinishesMay finish its side effects; result discarded
Cleanup (finally, compensations)RunsSkipped
Journal ends withrun:error, code="cancelled"run:error, code="terminated"
RetryableNoYes

Cancel is a request. With no worker for the workflow and version, or a step that hangs while its heartbeat keeps renewing, a cancelled run never ends; use terminate. Once latched, cancel wins: a later step failure or timeout still ends the run cancelled.

await client.cancel("fulfill_order", run_id, reason="wrong customer")
Terminal window
pyrula runs cancel <run_id> --reason "wrong customer"
curl -X DELETE "$BASE/fulfill_order/stream/<run_id>?reason=wrong%20customer"

The reason lands on the run:cancel event and the terminal’s detail. First reason wins.

result = await client.terminate("fulfill_order", run_id, reason="worker fleet lost")
result.outcome # TerminateOutcome.TERMINATED, or ALREADY_ENDED on a repeat
result.children # CloseResult(cancelled=0, terminated=2, abandoned=1)
result.run_id # the chain's live tail, if the run continued as new
result.ended # (name, run_id) of each child this call terminated
Terminal window
pyrula runs list --workflow fulfill_order --status in_progress --since 1h
pyrula runs show <run_id>
pyrula runs terminate <run_id> --reason "worker fleet lost" # asks first; --yes skips
curl -X POST $BASE/fulfill_order/runs/<run_id>/terminate -d '{"reason": "worker fleet lost"}'

handle.terminate(reason=...) does the same for a run you hold; AgentClient.terminate(agent, turn_id, reason=...) posts to /{agent}/turns/{turn_id}/terminate.

The store writes the terminal event and deletes the run’s lock atomically, so no worker needs to be connected. pyrula runs terminate talks straight to the store (--store or PYRULA_STORE_URL), with no runtime or Cloud. It exits 1 for an unknown run and 2 when it needs confirmation and has no terminal.

The journal gets one run:error with code="terminated"; detail holds terminated_at, reason (cut to 500 characters), requested_by and, for a child ended with its parent, a cause. Keep secrets out of reasons.

The store needs the terminate capability (Memory, SQLite and Valkey have it), else TerminateUnsupported is raised before anything is written and the HTTP route answers 501.

The deleted lock fences the run: a worker still running it, or returning after a crash, has every write refused, and its next heartbeat stands it down. A step already running can’t be interrupted, so its side effects still happen and its result is discarded.

client.retry or pyrula runs retry serves every step recorded before the terminate and reruns the step that was in flight, so a non-idempotent write in flight may run twice (see the retry’s in_doubt list). A run terminated while waiting on a child retries from before the first open child, which starts its own child. A run that was cancelled first can’t be retried.

On a graceful end the worker applies each open child’s parent_close policy (see Parallel steps and child runs). terminate has no worker, so the store does it:

  1. The parent is terminated first, so it starts no more children.
  2. abandon children are left running; cancel and terminate children are terminated.
  3. Each terminated child’s own open children get the same treatment, depth first.

A child whose submit hasn’t reached the store yet gets a held cancel and starts cancelled when the submit lands. An unclaimed hold expires after completed_ttl on Valkey (24 hours by default) or held_ttl on MemoryStore and SQLiteStore.

Each ended child carries requested_by="parent" and a cause naming the parent run. Repeating terminate on a terminated run reapplies the child policies, which finishes a cascade a crash cut short. On a run that ended any other way it writes nothing and returns ALREADY_ENDED.

A runtime connected to Cloud offers the turn.terminate control when its store has the terminate capability. The message names the run, an optional options.reason and options.requested_by; the journal records that actor, or cloud if none is given.

Because the store writes the terminal, the connector sends it to Cloud itself before acknowledging: the run, the live tail of a continued chain, and each terminated child. The pyrula runs terminate command has no Cloud connection, so a row it leaves open settles on the next terminal sweep of a runtime that holds the store.

A terminated run reaches Cloud as run_error with code terminated and a detail object. The reason is redacted like any failure detail and ships at capture io and above. Below io the actor ships only when it is the runtime’s own (cloud, parent, api or cli). A child cancelled because its parent ended reaches Cloud as cancel with a detail holding cancelled_at, reason, requested_by and cause.

View this page as Markdown