crystal-lang-consumer-kafka
You cannot run it on Mac with Container
https://github.com/crystal-lang/crystal/issues/11761
https://github.com/crystal-lang/distribution-scripts/issues/125
Workaround
brew install crystal librdkafka
docker compose -f docker-compose.local.yml up -d
cd crystal
shards install
Build
crystal build consumer.cr \
--release \
--no-debug \
-o consumer
Run
./consumer
x86/64
Bench
docker compose exec -it postgres psql -U bench -d bench -c "SELECT event_type, COUNT(*) as total_events, MIN(id) as first_id, MAX(id) as last_id, MIN(event_ts) as first_event_time, MAX(event_ts) as last_event_time, MIN(created_at) as first_created, MAX(created_at) as last_created FROM kiosk_events GROUP BY event_type ORDER BY event_type;"
output:
event_type |total_events|first_id|last_id|first_event_time |last_event_time |first_created |last_created |
-------------+------------+--------+-------+-----------------------+-----------------------+-----------------------+-----------------------+
crystal-visit| 1499001| 1|1499001|2024-11-27 22:33:10.728|2025-11-26 22:37:12.403|2025-11-26 22:38:19.131|2025-11-26 22:38:34.185|
ruby-visit | 1500000| 1499002|2999001|2025-11-26 22:41:46.000|2025-11-26 22:42:08.000|2025-11-26 22:41:46.000|2025-11-26 22:42:08.000|
Time Summary - Processing ~1.5M events
Crystal Consumer
- Total events: 1,499,001
- Processing time: ~15 seconds
- Start: 22:38:19
- End: 22:38:34
- Rate: ~99,933 events/second
Ruby Consumer
- Total events: 1,500,000
- Processing time: ~22 seconds
- Start: 22:41:46
- End: 22:42:08
- Rate: ~68,181 events/second
Performance Comparison
- Crystal is ~46% faster than Ruby
- Crystal processed the same amount of events in 7 seconds less
- Throughput ratio: Crystal is 1.46x faster
Time Summary - Processing ~50.5M events
event_type |total_events|first_id|last_id |first_event_time |last_event_time |first_created |last_created |
-------------+------------+--------+---------+-----------------------+-----------------------+-----------------------+-----------------------+
crystal-visit| 50500000| 1| 88421000|2024-11-27 22:51:17.266|2025-11-26 22:56:09.600|2025-11-26 22:51:19.709|2025-11-26 23:00:40.427|
ruby-visit | 50500000| 726001|101000000|2025-11-26 22:51:27.000|2025-11-26 23:03:50.000|2025-11-26 22:51:27.000|2025-11-26 23:03:50.000|
Crystal Consumer
- Total events: 50,500,000
- Processing time: ~9 minutes 21 seconds (561 seconds)
- Start: 22:51:19
- End: 23:00:40
- Rate: ~90,016 events/second
Ruby Consumer
- Total events: 50,500,000
- Processing time: ~12 minutes 23 seconds (743 seconds)
- Start: 22:51:27
- End: 23:03:50
- Rate: ~67,970 events/second
Performance Comparison
- Crystal is ~32% faster than Ruby
- Crystal processed the same amount of events in ~3 minutes less
- Throughput ratio: Crystal is 1.32x faster
Repository
crystal-lang-consumer-kafka
Owner
Statistic
- 0
- 0
- 0
- 0
- 0
- about 10 hours ago
- November 26, 2025
License
Links
Synced at
Wed, 26 Nov 2025 23:13:16 GMT
Languages