WeaveLog

A Crystal logging library that wraps the stdlib Log with rich terminal output via si_el_hay and structured JSON for production.

Drop-in compatible with Crystal's Log -- same Log.for, same Log::Entry, same backends. WeaveLog just makes it prettier and more powerful.

Features

• Pretty terminal output with severity icons, colors, source padding
• JSON (NDJSON) formatter for production log aggregation
• Pattern-based formatting ({timestamp} {severity} {message})
• File backend with size & time-based rotation + gzip compression
• Async backend with bounded queue and backpressure strategies
• Enrichers that auto-attach metadata (pid, hostname, fiber, environment)
• Backtrace buffer -- stores suppressed messages, dumps context on error
• Sampling and rate limiting per source
• Redaction of sensitive fields and regex patterns
• Dynamic runtime level control via signals
• Exception deduplication within time windows
• Syslog backend (RFC 5424, UDP/TCP)
• Log groups / sections with nested indentation
• Spec helpers with capture_logs and Spectator matchers
• Thread-safe (mutex-protected backends, atomic counters)

Installation

Add to your shard.yml:

dependencies:
  weave_log:
    github: alexanderadam/weave_log

Then run shards install.

Quick Start

require "weave_log"

# Development: pretty console output, debug level
WeaveLog.setup(:development)

log = Log.for("myapp")
log.info { "server started" }
log.debug &.emit("request", method: "GET", path: "/users", status: 200)
log.warn { "disk space low" }
log.error(exception: ex) { "request failed" }

# Production: JSON to stdout, info level
WeaveLog.setup(:production)

# From environment variables
# LOG_LEVEL=debug LOG_FORMAT=json LOG_SOURCES=myapp.*
WeaveLog.setup_from_env

Formatters

PrettyFormatter

Rich terminal output with icons, colors, and auto-padded source names.

fmt = WeaveLog::PrettyFormatter.new(
  severity_display: :icon,     # :icon, :label, or :both
  show_timestamp: true,
  timestamp_format: "%H:%M:%S.%3N",
  colorize: :auto,             # true, false, or :auto (detects $NO_COLOR)
  max_data_inline: 5,          # wrap data fields after N
  exception_style: :full,      # :full, :compact, or :message_only
  box_fatal: true,             # draw a box around fatal messages
  terminal_wrap: true,         # wrap long lines to terminal width
)

JSONFormatter

Single-line NDJSON for log aggregation (Datadog, Loki, ELK, etc.).

fmt = WeaveLog::JSONFormatter.new(
  timestamp_format: :iso8601,  # :iso8601, :epoch_ms, :epoch_s
  data_style: :flat,           # :flat merges into root, :nested under "data" key
  include_context: true,
)

PatternFormatter

Template-based formatting using {token} syntax.

fmt = WeaveLog::PatternFormatter.new(
  "{timestamp:%Y-%m-%d %H:%M:%S} [{severity:short}] {source} - {message} {data:json}"
)

Tokens: timestamp, severity (+ :icon, :short), message, source, data (+ :json), context, exception, pid, newline, fiber_id, fiber_name, hostname, env, caller, caller_file, caller_line.

RedactingFormatter

Wraps any formatter, scrubs sensitive data from output.

redactor = WeaveLog::Redactor.new(
  patterns: [WeaveLog::RedactionPatterns::CREDIT_CARD, WeaveLog::RedactionPatterns::SSN]
)
fmt = WeaveLog::RedactingFormatter.new(
  WeaveLog::PrettyFormatter.new,
  redactor
)

Default redacted fields: password, secret, token, api_key, authorization, credential, etc.

Backends

File Backend

backend = WeaveLog::FileBackend.new(
  path: "log/app.log",
  formatter: WeaveLog::JSONFormatter.new,
  flush_mode: :immediate,  # :immediate, :buffered, :periodic
)

Rotating File Backend

backend = WeaveLog::RotatingFileBackend.new(
  path: "log/app.log",
  formatter: WeaveLog::JSONFormatter.new,
  policy: :size,           # :size, :daily, :hourly
  max_size: 10_000_000,    # 10MB
  max_files: 5,
  compress: true,          # gzip old files in background
)

Async Backend

Non-blocking writes with bounded queue.

inner = WeaveLog::FileBackend.new(path: "log/app.log")
backend = WeaveLog::AsyncBackend.new(
  inner,
  queue_size: 1024,
  strategy: :drop_newest,  # :block, :drop_oldest, :drop_newest
)

Syslog Backend

RFC 5424 format via UDP or TCP.

backend = WeaveLog::SyslogBackend.new(
  host: "127.0.0.1",
  port: 514,
  transport: :udp,     # :udp or :tcp
  facility: :local0,
  app_name: "myapp",
)

Enrichers

Auto-attach metadata to every log entry.

WeaveLog.setup(:development, enrichers: [
  WeaveLog::ProcessInfo.new,          # pid, hostname
  WeaveLog::FiberInfo.new,            # fiber_id, fiber_name
  WeaveLog::EnvironmentInfo.new,  # crystal_env
  WeaveLog::OpenTelemetry.new,        # trace_id, span_id from context
  WeaveLog::CallerInfo.new,           # caller file:line (via wlog macro)
])

Caller Info (via macro)

log = Log.for("myapp")
WeaveLog.wlog(log, :info, "hello", user_id: 42)  # auto-injects __FILE__, __LINE__

OpenTelemetry Context

Reads trace_id/span_id from Log.context (set by your middleware):

Log.with_context(trace_id: "abc", span_id: "def") do
  log.info { "traced request" }  # enricher adds trace_id, span_id to data
end

Advanced Features

Backtrace Buffer

Stores suppressed (below-level) messages in a ring buffer. Auto-dumps context when an error occurs.

buffer = WeaveLog::BacktraceBuffer.new(
  inner: backend,
  buffer_size: 64,
  trigger_level: Log::Severity::Error,
)

Sampling

Log 1 in N messages per source.

sampled = WeaveLog::SamplingBackend.new(backend, ratio: 10)  # 1 in 10

Rate Limiting

Cap messages per source per time window.

limited = WeaveLog::RateLimitedBackend.new(
  backend,
  max_per_window: 100,
  window: 60.seconds,
)

Dynamic Level Control

Change log levels at runtime without restart.

WeaveLog::LevelControl.set_level("noisy.source", Log::Severity::Warn)
WeaveLog::LevelControl.reset_all

Signal handlers: SIGUSR1 cycles levels, SIGUSR2 dumps current config.

Exception Deduplication

Groups identical exceptions within a time window.

dedup = WeaveLog::DeduplicationBackend.new(
  backend,
  window: 30.seconds,
  group_by: :class_and_message,  # or :class_only
)

Stopwatch

Timed logging with automatic elapsed_ms.

sw = WeaveLog::Stopwatch.new("http.request")
# ... do work ...
sw.info &.emit("done", path: "/users")  # includes elapsed_ms in data

# Block form
WeaveLog.timed("db.query", "SELECT users") { query.execute }

Span / Trace

Structured tracing a la Rust's tracing crate. Spans nest, share a trace_id, and auto-set Log.context so all messages within a span are correlated.

WeaveLog.span("http.request", "GET /users") do |span|
  span.set(method: "GET", path: "/users", user_id: 42)

  WeaveLog.span("db.query", "SELECT users") do |inner|
    # inner span shares trace_id, gets its own span_id
    inner.event("rows fetched", count: 150)
  end

  # produces: start event, nested span events, finish event with elapsed_ms
end

Exception handling is automatic -- if the block raises, the span records the error and re-raises:

WeaveLog.span("risky.op") do |span|
  raise "oops"  # span finishes with status=error, logs the exception
end

WeaveLog.current_span returns the active span (or nil). Works with the existing {trace_id} / {span_id} pattern tokens and the OpenTelemetry enricher.

Conditional Logging

Filter entries at the backend level -- no if/else at the call site.

# only log DB entries with slow queries
backend = WeaveLog::ConditionalBackend.new(inner) do |entry|
  entry.data[:duration_ms]?.try(&.to_s.to_f64) { 0.0 } > 100.0
end

Pre-built conditions compose with AND/OR/NOT:

cond = WeaveLog::Conditions.all(
  WeaveLog::Conditions.source("api.*"),
  WeaveLog::Conditions.min_severity(Log::Severity::Warn),
)
backend = WeaveLog::ConditionalBackend.new(inner, &cond)

Available conditions: source, has_data, has_exception, min_severity, message_matches, all, any, not.

Log Groups / Sections

Nested, indented log sections with depth tracking.

WeaveLog.group("http.request") do
  log.info { "processing request" }     # indented 1 level in pretty output

  WeaveLog.group("db.query") do
    log.debug { "SELECT * FROM users" }  # indented 2 levels
  end
end

Groups automatically:

• Track fiber-local nesting depth
• Set group and group_depth in Log.context for structured output
• Indent lines in PrettyFormatter based on depth
• Log start/end events with group metadata
• Clean up on exceptions (ensure block)

WeaveLog.group_depth          # => current nesting depth (0 at top level)
WeaveLog.current_group        # => innermost group name, or nil
WeaveLog.group_indent         # => indentation string for current depth

Testing

Spec Helpers

require "weave_log/spec_helpers"

# Capture log entries during a block
logs = WeaveLog.capture_logs do
  Log.for("test").info { "hello" }
end

logs.has_entry?(severity: Log::Severity::Info, message: "hello")  # => true
logs.entries_matching(source: "test")                               # => [...]

Memory Backend

backend = WeaveLog::MemoryBackend.new
# ... use in tests ...
backend.size          # => 3
backend.last.message  # => "last message"
backend.clear!

Configuration

Block Form

WeaveLog.setup do |c|
  c.bind("myapp.*", Log::Severity::Debug, WeaveLog::PrettyBackend.new)
  c.bind("db.*", Log::Severity::Info, WeaveLog::JSONBackend.new)
end

Environment Variables

Migrating from stdlib Log

WeaveLog is a drop-in enhancement. Replace your backend setup:

# Before
Log.setup(:info, Log::IOBackend.new)

# After
require "weave_log"
WeaveLog.setup(:development)
# or
WeaveLog.setup(:production)

Your existing Log.for("source"), log.info { }, log.error(exception:) calls all work unchanged.

Performance Tuning

Choose the Right Backend Dispatch Mode

• Development: Use Log::DispatchMode::Sync (or WeaveLog.setup(:development)) so output appears immediately. Slower, but you see logs the instant they happen.
• Production: Use Log::DispatchMode::Async (default for WeaveLog.setup(:production)) so logging doesn't block your request path.

Async Backend for File I/O

Wrap file backends in AsyncBackend to move disk writes off the hot path:

file = WeaveLog::FileBackend.new("app.log")
async = WeaveLog::AsyncBackend.new(file, queue_size: 4096, strategy: :drop_newest)

Choose a backpressure strategy:

• :block -- safest, no log loss, but pauses caller when queue is full
• :drop_newest -- never blocks, drops new messages under pressure
• :drop_oldest -- never blocks, drops oldest queued messages

Buffered Writes

Use flush_mode: :buffered or :periodic for FileBackend to batch disk writes. Errors always flush immediately.

Sampling & Rate Limiting

For high-throughput sources, sample or rate-limit to reduce volume:

WeaveLog.setup(:production, sampling_ratio: 10)  # log 1 in 10
WeaveLog.setup(:production, rate_limit: {max: 100, window: 60.seconds})

Formatter Choice

JSONFormatter is faster than PrettyFormatter since it skips ANSI escape codes and alignment tracking. In production, always use JSON.

Suppressed Messages

Messages below the configured level are cheap -- they don't allocate an IO or call the formatter. Crystal's Log blocks aren't even evaluated:

log.debug { expensive_computation }  # block never called if level > debug

Benchmarks

Run the built-in benchmarks to measure throughput on your hardware:

crystal run src/weave_log/benchmark.cr --release

Development

shards install
crystal spec             # run all 386 specs
crystal tool format src/ # check formatting
./bin/ameba              # lint

Contributing

1. Fork it (https://github.com/alexanderadam/weave_log/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

License

MIT

Contributors

• Alexander Adam - creator and maintainer