Choosing a store

The store holds every run’s journal, leases, timers, signals, schedules and idempotency keys. Workers, clients, the dev UI and the CLI for one environment share one, named by PYRULA_STORE_URL (or --store). open_store() (pyrula.workflows) and open_agent_store() (pyrula.agents) build it from the URL.

	Memory	SQLite	Valkey	Valkey + Postgres archive
URL	memory:// (pyrula dev and tests only)	sqlite:////var/lib/pyrula/pyrula.db	valkey://host:6379/0	the Valkey URL plus PYRULA_POSTGRES_DSN
Survives a restart	No	Yes	Yes, with AOF or RDB persistence	Yes; finished runs also in Postgres
Hosts	One process	One host, several processes	Any number	Any number
Something to run	Nothing	Nothing	A Valkey 7.2+ or Redis 6.0+ server	Valkey and Postgres 15+
Built for	Tests, throwaway pyrula dev	Development, demos, CI, low traffic on one host	Production, several hosts	Production with a long run history
Finished runs kept	Until the process exits	7 days (retention_seconds)	1 day (completed_ttl)	In Postgres, until you delete them
Backups	None	pyrula store backup DEST	Valkey persistence and replication	Valkey, plus Postgres backups
Refuses	memory:// outside pyrula dev and tests	A second host, network filesystems, Windows, SQLite before 3.37	An allkeys-* eviction policy while maxmemory is set (volatile-* only warns)	As Valkey

Valkey is the production store, and the only one for several hosts, sustained load or replication. Add the Postgres archive when finished runs must outlive Valkey’s retention.

SQLite

pyrula dev defaults to SQLite in .pyrula/dev.db. Workers take the same URL form (sqlite:///relative.db, sqlite:////absolute.db):

export PYRULA_STORE_URL=sqlite:////var/lib/pyrula/pyrula.db
pyrula worker --workflow app:process_order

Query parameters: synchronous=full|normal and retention_seconds=<seconds>|none; SQLiteStore(path, retention_seconds=..., synchronous=...) takes the same. The directory is created 0700 and the file 0600; nothing in it is encrypted. A one-host Compose recipe is in Deploy a worker.

One host

Every process that opens the file must run on the same machine. The store refuses, on open:

• A network or shared filesystem (NFS, SMB, 9p, virtiofs, FUSE and similar).
• A second kernel while the first was seen within 3 x lock_ttl (90 s). Containers on one Docker host share a kernel; a macOS process and a Docker Desktop container on one bind mount do not.
• Windows. Use WSL2 or Valkey. pyrula dev alone falls back to memory:// there, with one line saying so.
• A SQLite library older than 3.37.0.

Leases use the host’s clock, so a large wall-clock jump expires leases early.

One writer at a time

SQLite allows one writer, so write throughput is set by commits, not worker count. The worker buffers a run’s events and writes them in one transaction (BEGIN IMMEDIATE) at each step end, at the end of the run and every 64 events. A run also commits when a worker claims it and renews its lease. With synchronous=full each commit waits for a disk sync, and every worker process on the host shares that one commit stream.

Plan on roughly 300 to 1,000 commits a second per host on network-attached cloud disks such as EBS gp3. That figure is estimated from sync latency, not yet measured. A run of five steps needs about eight commits (also an estimate), so it caps you near 40 to 120 such runs a second however many workers you add. synchronous=normal skips the sync per commit and raises the ceiling; by how much depends on your disk, and it is not yet measured either. benchmarks/core/bench_sqlite_store.py times both settings on your disk and prints the cost of one sync. Past the ceiling, use Valkey.

A waiting writer retries every 0.5 to 2 ms until busy_timeout_s (5 s). A writer that never lets go, such as a long transaction from another tool, makes lease heartbeats fail (lock_ttl 30 s, every 10 s); the leases lapse and another worker recovers the runs as after a crash. After a crash, runs resume after lock_ttl plus up to one orphan_scan_interval: 30 to 90 s by default.

Durability

The store runs in WAL mode: pyrula.db, pyrula.db-wal and pyrula.db-shm together are the store.

• synchronous=full (default) syncs the log on every commit, so a committed step survives power loss on a disk that honours fsync. macOS doesn’t: Apple’s fsync doesn’t flush the drive cache. Run production on Linux.
• synchronous=normal survives kill -9 but can lose the last commits on power loss or an OS crash, whatever they were: a submit() that returned (the run never exists), a signal or cancel that was acknowledged, a step result, a run:complete the caller already saw. With full, an acknowledged commit survives kill -9 and a process crash too. Choose normal only when callers can resubmit, steps are idempotent and a lost signal or completion is tolerable, such as development, CI and demos. Set it with ?synchronous=normal on the store URL or SQLiteStore(..., synchronous="normal").

Copying pyrula.db alone is not a backup. pyrula store backup DEST writes a consistent copy while workers run, and pyrula store info prints size, log size, retention, runs by status and the hosts that opened the file. There is no replication.

Retention

Finished runs are deleted after 7 days, journal and all. Change it with retention_seconds (or ?retention_seconds=), or none to keep them. A run in an open continue_as_new chain stays until the chain closes. What-if and replay need the journal, so they work only while the run is retained.

Plan on roughly 1 KB of disk per event with payloads of a few hundred bytes: 10 runs of 10 events a second is about 60 GB at 7 days.

Not on SQLite

• A Postgres archive: --postgres-dsn with SQLite is refused.
• Shards: the queue is one indexed table, so the shard options are unused.
• Moving live runs to another store: drain the workers, then repoint them. Finished runs move as saved cases.

Valkey

Valkey 7.2+ or Redis 6.0+ holds the live run in memory on a server any number of hosts share, and workers spread a workflow’s queue across shards. Run it with persistence and maxmemory-policy noeviction (per managed service). See the production checklist, Installation for connection options and Deployment modes for the Postgres archive.