Known limitations
Upgrades within 0.1.x
Section titled “Upgrades within 0.1.x”A patch doesn’t change the run journal or the Cloud wire format, so a run started on one 0.1.x worker resumes on a later one. A minor (0.2) may change a stored format and lists every change (Versioning & stability).
The Valkey key layout isn’t versioned. When release notes list a layout change, drain in-flight runs or use a fresh namespace.
Runtime
Section titled “Runtime”- Python 3.11 to 3.15, GIL builds only. Importing
pyrulaonpython3.13tor later raisesImportError. - The
litellmextra doesn’t install on 3.15 yet (fastuuid has no 3.15 build). - Python only. Traces from other languages reach Cloud through OpenTelemetry.
- Windows: the SQLite store refuses it, so
pyrula devkeeps its history in memory there (use WSL2 or Valkey to keep it). Wheels are built for Windows ARM64 but not run in CI; the Linux, macOS and Windows x86_64 wheels are smoke-tested. - A
@workflowbody must beasync def, andWorkflowClientand the agents client are async. A step or tool may be sync, but not with bothtimeout=andretry=. Sync code usespyrula.workflows.sync.WorkflowClient(From sync code); the agents client has no sync form.
Stores
Section titled “Stores”- Several hosts or real load need Valkey 7.2+ or Redis 6.0+ with
maxmemory-policy noeviction. SQLite refuses a second host and network filesystems (Choosing a store). - SQLite on macOS isn’t power-safe:
fsyncdoesn’t flush the drive cache. Run production on Linux. - Postgres is an archive, not a live store.
- A run ends with
stream_too_largepast 50,000 events (max_event_count_per_run). On Valkey the cap never exceeds what the store keeps per run (maxlen, 100,000, less a margin), so the history stays whole. Usectx.continue_as_new()for longer-lived runs (History limits). - One event payload is capped at 1,000,000 bytes; a bigger one ends the run with
PayloadTooLarge.max_event_payload_bytes=Noneon@workflowlifts it. list_runspages one workflow’s runs, newest first. There’s no search across workflows or by field. On Valkey an owner-filtered page can come back short: page until the cursor isNone.
Ceilings
Section titled “Ceilings”Where a number below is an estimate, it says so. The estimates have not been measured under load yet.
| What | Ceiling | Set by |
|---|---|---|
| SQLite commits per host | About 300 to 1,000 a second on EBS gp3 with synchronous=full. Estimated from sync latency. | The disk’s sync rate; synchronous=normal raises it (One writer at a time) |
| Live runs per worker process | max_concurrent_runs, 50 by default. Each live run holds one OS thread for its lease renewal. | You, in WorkerOptions |
| Events per run | 50,000 | max_event_count_per_run (max_event_count_per_turn for agents), lowered to what the store keeps: about 99,000 on Valkey even with None |
| One event payload | 1,000,000 bytes | max_event_payload_bytes |
| Cloud ingest requests per runtime | 1,200 a minute (20 a second) | Cloud |
| Cloud ingest requests per project | 12,000 a minute (200 a second) | Cloud |
| Connector send rate per process | About 850 events a second at 4 KB an event and 150 ms to Cloud. Estimated: the connector sends one batch at a time. | Round-trip time and the 512 KiB batch limit |
Lease-renewal threads
Section titled “Lease-renewal threads”Each live run renews its lease from its own OS thread, every heartbeat_interval (10 s by
default). A process with 50 slots runs 50 such threads, which is nothing. The threads start to
matter in the high hundreds of slots per process, an estimate from the thread count and not a
measurement. With many concurrent runs, keep max_concurrent_runs modest and start more worker
processes, each with its own slots, instead of raising one process’s limit
(Worker).
Cloud ingest rates
Section titled “Cloud ingest rates”Cloud limits the requests a runtime and a project may make, not the events in them. A request
over a limit gets a 429 with Retry-After, and the connector backs off and retries without
losing spooled events (Connect to Pyrula Cloud).
The connector sends a batch when 500 events are waiting, after a one-second wait, or 15 ms after a terminal or suspend event, and it sends one batch at a time. A runtime emitting a trickle of events sends about 60 requests a minute. A saturated one sends one request per round trip: about 400 a minute at 150 ms, and more than the 1,200 allowed under about 50 ms with many short runs. Those are estimates from the connector’s flush rules, not measurements. At 150 ms about 30 saturated runtimes reach the project limit; a project of mostly idle runtimes carries about 200. Sustained event throughput per project is bounded by Cloud’s database, not by these limits, and has not been measured. A run over the monthly run quota still executes on your worker, but Cloud stores none of its events.
Runs, timers, timeouts
Section titled “Runs, timers, timeouts”- Timeouts are per attempt:
@workflow(timeout=...)(300 s),max_attempt_wall_secondsand a step’stimeout=. There is no whole-run deadline. - Timeouts need the event loop. A blocking call inside
asynccode stops them firing; pastmax(3 × lock_ttl, timeout)the attempt stands down, and afterpoison_max_retriesof those the run is quarantined withdetail.cause: loop_blocked. - Sleeps and wait deadlines fire on the worker’s timer poll (5 s,
timer_poll_interval_s); schedules are checked every 30 s. - Cancel stops at the next step boundary.
terminateends the run now, but a running step may still finish its side effects (Cancel and terminate). - Side effects are at least once: a step running when its worker died runs again.
Exactly-once holds only for Kafka output bound to its offset commit with
delivery="exactly_once"(Replay and recovery). - A crash or lost lease restarts the retry count at attempt 1 (Errors and retries).
- An agent turn can’t be retried (
retry_agent_run): submit it again (Retrying a failed run). ctx.run_childpolls the child while holding the parent’s worker slot.
Schedules
Section titled “Schedules”After downtime a schedule fires at most once for the backlog (catchup_window limits even
that). There is no schedule history view and no declarative @workflow(schedule=...).
buffer_one overlap is reserved, not implemented (Catch-up).
Agents
Section titled “Agents”LLMRatemeters the calls anAgentLLMmakes (run,stream,complete,structured) and every retry, whichAgentLLMmakes for the OpenAI, LiteLLM and Anthropic clients. Thepyrula.agents.compatclients anddurable_chat_modeltakerate_limittoo, but only meter where a run’s store is reachable (a call outside a run is not limited), anddurable_chat_modelcannot meter its inner model’s own retries. Pydantic AI model calls underPyrulaDurabilityignoreLLMRateentirely.MemoryStorelimits one process andSQLiteStoreone host, so useValkeyStoreto share a limit across hosts.- Embedded mode (an agent under
pyrula dev, orcreate_appwith the defaultdeployment_mode="embedded") runs no timer or child-claim loop. It refusesctx.sleep,ctx.sleep_until,ctx.run_child,ctx.spawn_workflow,ctx.gather_subagents, waits withtimeout=ordeadline=, and tools withapproval_timeout. Run a worker for those (details). - Text arrives in batches, not token by token. With
StreamingMode.SAFEa call over 5 MiB of output ends the turn withBUFFER_OVERFLOW(Streaming mode). - Spend caps can overshoot by one in-flight call per concurrent caller. A plain
@workflowcalling a model through a drop-in isn’t metered (Default prices).
Integrations
Section titled “Integrations”- OpenAI:
chat.completions.createandresponses.createare journaled inside a run, streaming or not, and so are theparseandstreamhelpers built on them. Background responses,conversation=,responses.retrieve/cancel/compact/connectand hosted tools that run code, call remote MCP servers or generate images are refused. A call that runs live after a resume or what-if needs itsprevious_response_idresponse still stored at OpenAI (drop-in). - OpenAI Agents SDK: the model calls replay through the journaled client, and nothing else does
(recipe). A function, computer or local-shell tool
body outside a
@stepruns again on a resume. The Responses WebSocket transport,LitellmModel, any-llm models, Realtime and voice bypass the client and are not journaled, with no error.Sessionmemory makes a resume raiseReplayInputChanged, aneeds_approvalinterruption is not a durable pause, and trace export is sent again on a replay. The SDK (0.21 and later) needsopenai3, so it can’t share an environment withpyrula[litellm]until litellm lifts itsopenai<3cap. It is tested onopenai-agents0.24 andopenai3.41. - OpenAI chat and Anthropic streams are journaled when the terminal chunk arrives. A stream you stop reading earlier runs live again on resume (streams).
- MCP: streamable HTTP only,
mcp1.x, tools and text results, one call at a time per server (MCP limits). - LangGraph: a graph can’t use a checkpointer or its own
interrupt()(Not supported). - Pydantic AI: approval,
@tool(writes=...)and per-tool limits don’t apply to its tools, tool calls run one at a time, andagent.run(toolsets=)is refused (What does not apply). - Kafka: the confluent-kafka-compatible client refuses unsupported keys, such as the KIP-848
consumerprotocol and PKCS#12 keystores (Confluent compatibility).
What-if and replay
Section titled “What-if and replay”- A what-if needs the source run’s journal. Finished runs are deleted after
completed_ttl(24 h) on Valkey orretention_seconds(7 days) on SQLite. Keep one with the Postgres archive or by saving it as a case (What-if limits). - What-ifs start from Pyrula Cloud: no
client.whatif, no CLI command. Offline,pyrula case run --inputreplays a saved case with changed input. - CI replay checks that the code makes the same calls, not that the output is good (Replay in CI).
Pyrula Cloud
Section titled “Pyrula Cloud”- Over the monthly run quota a run still executes on your worker, but Cloud stores none of its
events. The connector logs
cloud_ingest_run_rejectedwithreason=runs_quota_exceeded. developmentandstagingkeys default to capture levelio;productionand every other environment default tolifecycle(What leaves your runtime).