# Benchmarks

> pyrula-kafka throughput vs confluent-kafka, aiokafka, and kafka-python.

Source: https://docs.pyrula.io/kafka/benchmarks/

Records per second, higher is better. Local `confluentinc/cp-kafka:7.7.0`, Apple M1 Max,
12 partitions, median of three runs; rows move 10-20% between runs. pyrula rows use the bulk
produce and batch poll APIs. Bold is the faster of pyrula and confluent. No Linux run is
published yet.

The harness is `benchmarks/kafka/benchmark.py`; run it against your own broker:

```bash
python benchmarks/kafka/benchmark.py --broker localhost:9092
```

`--help` lists the suites and sizes; the Avro suites need `--schema-registry`.

## 100,000 records, 1 KB

Near the broker's ceiling (~155-175 MB/s), so the faster clients converge.

| Suite | pyrula | confluent | aiokafka | kafka-python | vs confluent |
| --- | --- | --- | --- | --- | --- |
| Produce bytes | 140k | 144k | 84k | 6.1k | 0.99x |
| Produce bytes, acks=1 | **150k** | 146k | n/a | n/a | **1.02x** |
| Idempotent produce, 6 partitions | **157k** | 149k | n/a | n/a | **1.09x** |
| Produce JSON | **125k** | 115k | 77k | 5.4k | **1.08x** |
| Transactional produce | **91k** | 64k | n/a | n/a | **1.41x** |
| Produce Avro | **128k** | 125k | n/a | n/a | **1.02x** |
| Consume records | **138k** | 126k | 103k | 6.4k | **1.02x** |
| Consume raw bytes | **165k** | 156k | n/a | n/a | 1.00x |
| Consume raw bytes, 12 partitions | 146k | 156k | n/a | 6.4k | 1.04x |
| Consume + JSON decode | **137k** | 131k | 82k | 5.8k | **1.19x** |
| Consume Avro | 108k | 110k | n/a | n/a | 1.00x |

## 1,000,000 records, 128 B

JSON and Avro rows encode a small dict, so records are under 128 B.

| Suite | pyrula | confluent | aiokafka | kafka-python | vs confluent |
| --- | --- | --- | --- | --- | --- |
| Produce bytes | **1.12M** | 1.08M | 194k | 31k | **1.04x** |
| Produce bytes, acks=1 | **1.06M** | 962k | n/a | n/a | **1.10x** |
| Idempotent produce, 6 partitions | 943k | 968k | n/a | n/a | 0.98x |
| Produce JSON | **1.07M** | 403k | 189k | 34k | **2.75x** |
| Transactional produce | **210k** | 138k | n/a | n/a | **1.50x** |
| Produce Avro | **1.87M** | 427k | n/a | n/a | **4.43x** |
| Consume records | **769k** | 731k | 474k | 41k | **1.06x** |
| Consume raw bytes | **939k** | 907k | n/a | n/a | **1.06x** |
| Consume raw bytes, 12 partitions | 872k | 875k | n/a | 41k | 1.04x |
| Consume + JSON decode | **1.24M** | 113k | 299k | 50k | **10.2x** |
| Consume Avro | **1.04M** | 144k | n/a | n/a | **7.7x** |

`n/a`: not measured (kafka-python has no transaction API).