Kafka runtime guarantees

The Kafka runtime provides exactly-once processing and at-least-once output delivery by default. Each matrix cell maps to a conformance test. @kafka_agent uses the same runtime contracts as @kafka_workflow with an additional LLM loop.

yes means a conformance contract asserts the guarantee holds; partial means it holds within the stated boundary; no means a test asserts the opposite and the row says why.

Binding	processing	output delivery	poison	handler error
kafka_workflow	yes	partial	yes	yes
kafka_workflow[exactly_once]	yes	yes	yes	yes
kafka_workflow[durable=False]	no	partial	yes	yes
kafka_agent	yes	partial	yes	yes
kafka_agent[exactly_once]	yes	yes	yes	yes

Column definitions

• processing: Exactly-once: a redelivered record whose run completed is not re-run.
  • Its result is read back from the run store.
  • A retried record whose run is partial resumes from its checkpointed steps; completed steps do not re-execute.
  • durable=False rows are at-least-once: the handler and every inline step re-run.
• output delivery: A failed delivery never commits past its record.
  • at_least_once rows: a failed delivery commits nothing, so the record is redelivered and re-produced.
  • exactly_once rows: output records and input offsets commit in one transaction; an abort leaves neither.
• poison: A record that fails decode goes to the DLQ topic, with the error, before the workflow runs.
  • The run store never sees it; the rest of the batch proceeds and commits.
• handler error: After max_retries, the on_error disposition holds.
  • dlq routes the record with the error; fail stops the worker without committing past it; skip commits past it.
  • A run that pauses (ctx.sleep, ctx.interrupt, ctx.wait_for_signal) is an error on this path and never stalls its partition.

Install and tested versions

All bindings: built in. Tested against confluentinc/cp-kafka 7.7.0 (floor); also proven: apache/kafka:4.0.0 (apache-kafka, cluster, 2026-10-04), apache/kafka:4.3.1 (apache-kafka, cluster, 2026-10-04), docker.redpanda.com/redpandadata/redpanda:v25.1.12 (redpanda, single, 2026-10-04).

Notes

• kafka_workflow[durable=False]: durable=False has no run store: the handler re-runs on every retry or redelivery, by design

Limits

• kafka_workflow - output delivery; kafka_workflow[durable=False] - output delivery; kafka_agent - output delivery: at-least-once: a crash between produce and commit re-produces on redelivery; dedupe downstream on key, or choose delivery=“exactly_once”

Also pinned, outside the matrix

These promises need more than the fakes-driven conformance harness - a real worker lifecycle, a mixed batch, a stacked decorator, or a live rebalance - so they are proven by a runtime test directly rather than by a matrix cell.

• Handler error: on_error="fail" stops the runner’s worker, not only the one record - pyrula-kafka/tests/broker/test_agents_runner_unit.py::test_on_error_fail_does_not_commit_and_stops.
• Processing: kafka_workflow(durable=False) runs with no run store and reruns on redelivery under the plain KafkaRunner, not only the harness’s KafkaAgentRunner - pyrula-kafka/tests/broker/test_kafka_workflow_durable_false.py::test_runner_durable_false_has_no_store_and_reruns.
• Poison: a poison record sharing a drain with good records does not stall the batch or take the consumer down with it - pyrula-kafka/tests/broker/test_runner_mixed_output_and_dlq.py.
• Processing: an existing @workflow stacked under @kafka_workflow still coerces its persisted input the same way a bare handler does - pyrula-kafka/tests/broker/test_kafka_workflow_stacked_decorator.py.
• Output delivery: exactly-once holds across a live rebalance - every input id on the output exactly once, read-committed - pyrula-kafka/tests/broker/test_agents_runner_live.py::test_eos_rebalance_no_loss_no_dup.