Karma

Karma is a small TCP service for fast, day-bucketed limit usage accounting.

Use it when an application needs fresh usage totals by limit, subject, and UTC day on the hot path. Karma keeps counters in memory, persists accepted writes with snapshots and a write-ahead log (WAL), and speaks newline-delimited JSON over TCP.

Russian documentation: README.ru.md.

When to use it

Karma is built for this shape of problem:

application receives a request
  -> records usage for a subject
  -> reads the current usage total for the same limit
  -> decides whether the subject is still within the limit

Good fits:

• API request, email, export, job, or storage usage limits;
• many subjects with small day-bucketed counters;
• fresh totals that are cheaper to read from a focused read model than from an analytical database;
• at-least-once producers that need idempotent writes.

Not a fit:

• arbitrary time-series tags or analytical queries;
• multi-master replication;
• automatic leader election or quorum writes;
• strong cross-node read-after-write guarantees from replicas.

Quick start

Requirements:

• Crystal 1.17.1
• Shards

Build and run a local node:

shards build --release
bin/karma \
  --bind=127.0.0.1 \
  --port=8080 \
  --directory=.karma-data \
  --restore=true \
  --wal=true

Write one usage event:

printf '{"v":2,"op":"counter.increment","series":"api_requests","key":42,"value":1}\n' \
  | nc 127.0.0.1 8080

Read the current total:

printf '{"v":2,"op":"counter.sum","series":"api_requests","key":42}\n' \
  | nc 127.0.0.1 8080

Successful responses use the v2 envelope:

{"protocol_version":2,"success":true,"response":1,"error_code":null}

Run with Docker:

docker build -t karma:local .
docker run --rm \
  -p 8080:8080 \
  -v karma-data:/data \
  karma:local \
  --bind=0.0.0.0 \
  --port=8080 \
  --directory=/data \
  --restore=true \
  --wal=true \
  --wal-fsync=true

For production, use a persistent volume, keep WAL enabled, keep --wal-fsync=true unless you have a measured reason not to, scrape metrics, and create regular snapshots with snapshot.create_all or SIGUSR1.

Core concepts

The public v2 protocol still contains some historical tree.* operation names. For new examples and clients, use limit-usage language: series, key, bucket, and value.

Read commands do not create missing series. A missing series returns not_found; a missing key inside an existing series returns zero or an empty result.

Protocol

Karma 1.0 accepts protocol v2 only.

• One request is one JSON object followed by \n.
• One response is one JSON object followed by \r\n.
• Every request must include "v": 2 and op.
• If --auth-token is set, requests must include token.
• If --read-auth-token is set, that token can run only read-only commands.

Example request:

{"v":2,"op":"counter.increment","series":"api_requests","key":42,"bucket":20260620,"value":1}

Example error:

{
  "protocol_version": 2,
  "success": false,
  "response": "Field tree or series is required",
  "error_code": "validation_error"
}

Stable error codes include invalid_json, unsupported_protocol, unknown_command, validation_error, not_found, unauthorized, forbidden, request_too_large, response_too_large, query_timeout, idempotency_conflict, replication_gap, replication_error, and internal_error.

Common operations

Create a limit explicitly when you want setup to be visible:

{"v":2,"op":"tree.create","series":"api_requests"}

Record and read usage:

{"v":2,"op":"counter.increment","series":"api_requests","key":42,"value":1}
{"v":2,"op":"counter.increment","series":"api_requests","key":42,"bucket":20260620,"value":10}
{"v":2,"op":"counter.decrement","series":"api_requests","key":42,"bucket":20260620,"value":1}
{"v":2,"op":"counter.sum","series":"api_requests","key":42}
{"v":2,"op":"counter.sum","series":"api_requests","key":42,"range":{"from":20260601,"to":20260620}}
{"v":2,"op":"counter.series","series":"api_requests","key":42,"range":{"from":20260601,"to":20260620}}

Use batches when the application can collapse events before sending them:

{"v":2,"op":"series.batch_add","series":"api_requests","items":[[42,20260620,37],[43,20260620,12]]}
{"v":2,"op":"series.batch_set","series":"api_requests","items":[[42,20260620,100],[43,20260620,0]]}
{"v":2,"op":"counter.batch_sum","series":"api_requests","keys":[41,42,43]}
{"v":2,"op":"counter.multi_sum","items":[{"series":"api_requests","key":101},{"series":"emails_sent","key":101}]}

series.batch_set writes exact bucket values. A zero value deletes that bucket. Large requests must fit --max-request-bytes.

Inspect and maintain a limit:

{"v":2,"op":"tree.list"}
{"v":2,"op":"tree.info","series":"api_requests"}
{"v":2,"op":"tree.keys","series":"api_requests","limit":1000,"cursor":0}
{"v":2,"op":"tree.summary","series":"api_requests","range":{"from":20260601,"to":20260620}}
{"v":2,"op":"tree.top","series":"api_requests","limit":100}
{"v":2,"op":"series.delete_before","series":"api_requests","before":20260601}
{"v":2,"op":"series.compact","series":"api_requests"}

Reset or delete usage when an application needs to remove stored buckets:

{"v":2,"op":"counter.reset","series":"api_requests","key":42}
{"v":2,"op":"counter.delete_range","series":"api_requests","key":42,"range":{"from":20260601,"to":20260620}}
{"v":2,"op":"tree.reset","series":"api_requests"}

Idempotent writes

Mutating commands can include idempotency_key. Karma stores the first successful response for that key. Repeating the same payload returns the saved response with "idempotent": true; reusing the key with a different payload returns idempotency_conflict.

{"v":2,"op":"series.batch_add","series":"api_requests","items":[[42,20260620,37]],"idempotency_key":"usage-offsets-1000-1499"}

Use idempotency for writes that may be retried by a queue, job runner, import, or at-least-once event source. Do not retry mutating writes blindly.

Idempotency retention is controlled by --idempotency-max-records, --idempotency-max-age-seconds, and manual pruning:

{"v":2,"op":"idempotency.prune","before":"2026-06-20T00:00:00Z","limit":10000}

Streaming ingest

Streaming ingest is for rebuilds, backfills, and large imports. It supports:

{"v":2,"op":"ingest.begin","stream_id":"import-20260620","mode":"add","granularity":"day"}
{"v":2,"op":"ingest.chunk","stream_id":"import-20260620","series":"api_requests","chunk_seq":1,"items":[[42,20260620,10]]}
{"v":2,"op":"ingest.commit","stream_id":"import-20260620"}

Duplicate chunks are skipped. Out-of-order chunks are rejected before they are applied. A committed stream is remembered durably, including after restart, snapshot restore, or replication bootstrap.

Persistence and recovery

Karma stores live counters in memory and persists data through:

• snapshots: MessagePack .tree files, one per series;
• WAL: newline-delimited JSON entries in karma.wal.

With --restore=true, startup loads the latest snapshot per series and replays WAL entries after the snapshot LSN.

Snapshot commands:

{"v":2,"op":"snapshot.create","series":"api_requests"}
{"v":2,"op":"snapshot.create_all"}
{"v":2,"op":"snapshot.list"}
{"v":2,"op":"snapshot.info"}
{"v":2,"op":"snapshot.verify"}

snapshot.create_all writes atomic snapshots, fsyncs them, truncates WAL after successful snapshotting, and prunes old snapshots according to --dump-retention-per-tree.

Recovery markers for external pipelines:

{"v":2,"op":"recovery.checkpoint","source":"usage-export","offset":"export-2026-06-20","event_id":"batch-42"}
{"v":2,"op":"recovery.status"}
{"v":2,"op":"reconciliation.report","checked_points":1000,"mismatch_count":2,"absolute_drift":15,"max_abs_delta":10}

Replication

Karma supports asynchronous master-to-slave replication. A slave bootstraps from master snapshots and then polls WAL entries.

bin/karma \
  --role=slave \
  --port=8081 \
  --directory=/var/lib/karma-slave \
  --restore=true \
  --replication-source-host=127.0.0.1 \
  --replication-source-port=8080 \
  --replication-token=read-secret

Useful checks:

{"v":2,"op":"replication.status"}
{"v":2,"op":"replication.entries","after_lsn":120,"limit":1000}

Replication boundaries:

• replication is asynchronous;
• slave nodes reject direct mutating client commands;
• failover is manual;
• the old master must be stopped before promoting a slave.

Runbook: docs/replication-operations-runbook.md.

Operations

Health and metrics:

{"v":2,"op":"system.ping"}
{"v":2,"op":"system.health"}
{"v":2,"op":"system.stats"}
{"v":2,"op":"system.metrics"}

Production runbook: docs/runbook.md.

Signals:

• SIGINT: stop accepting clients, snapshot all series, truncate WAL after successful snapshots, and exit with status 0.
• SIGUSR1: snapshot all series, truncate WAL after successful snapshots, and keep running.

Common startup options:

Run bin/karma --help for the full flag list.

Clients

Crystal:

dependencies:
  karma_client:
    path: clients/crystal

KarmaClient.with_client do |karma|
  karma.record_usage("api_requests", subject_id: 42, amount: 1, day: Time.utc)
  karma.usage("api_requests", subject_id: 42)
end

See clients/crystal.

Ruby/Rails:

gem "karma_client", path: "clients/ruby"

KarmaClient.with_client do |karma|
  karma.increment(series: "api_requests", key: 42, bucket: Date.current, value: 1)
  karma.sum(series: "api_requests", key: 42, from: 7.days.ago.to_date, to: Date.current)
end

See clients/ruby.

Performance

Local performance depends on CPU, disk, filesystem, runtime, network, and workload mix. Treat the included results as regression checks, not universal promises.

Recent local checks showed:

• in-process single reads and writes in the hundreds of thousands of ops/sec with WAL off;
• batched add and batched sum paths above one million items or keys per second in local profiles;
• persisted single increments becoming WAL-bound when WAL is enabled;
• 10M-key scalability checks using about 5 GiB of reserved heap for 70M daily data points in one local run.

Charts:

Development

Developer guide: docs/development.ru.md.

Baseline checks:

crystal spec
shards build --release

Focused suites:

crystal spec spec/command_spec.cr
crystal spec spec/wal_spec.cr
crystal spec spec/replication_spec.cr
crystal spec spec/idempotency_spec.cr
crystal spec spec/bucketed_counter
crystal spec clients/crystal/spec
ruby clients/ruby/test/karma_client_test.rb

Do not commit generated runtime data such as .crystal-cache-*, .karma-data, .spec_*, bin/, snapshots, WAL files, or temporary files.

Support and status

Current shard version: 1.0.1.

The supported public API is protocol v2. Legacy/v1 requests are intentionally not supported in Karma 1.0.

For a bug report, include the Karma version, startup flags, request JSON, response envelope, and relevant logs. For production triage, start with docs/runbook.md and, for replicas, docs/replication-operations-runbook.md.

License

MIT