wcwidth

Unicode-correct terminal display width for Crystal.

A Crystal port of shpeckman/wcwidth, measuring text the way a terminal actually renders it: by grapheme cluster, not by code point.

Table of Contents

• Overview
• Features
• Installation
• Usage
  • Whole-string width
  • Column helpers
  • One-shot measurement
  • Grapheme iteration
  • Streaming
• API Reference
• Configuration
• Behavior Notes
• Verification
• Performance
• License

Overview

String#size counts code points, which is the wrong unit for terminal layout. A family emoji is seven code points but occupies two columns; e followed by a combining acute is two code points but one column. Getting this wrong misaligns tables, corrupts progress bars, and leaves cursor artifacts.

This library segments text into grapheme clusters per UAX #29 and reports the display width of each.

"👨‍👩‍👧‍👦".size            # => 7
WCWidth.swidth("👨‍👩‍👧‍👦")   # => 2

Features

• Full UAX #29 grapheme cluster segmentation, including ZWJ emoji sequences, regional-indicator flag pairs, Hangul syllables, and Indic conjuncts (GB9c)
• Emoji presentation handling: VS15/VS16 variation selectors widen or narrow the base character
• Two-stage lookup tables generated from Unicode 17.0.0
• Streaming API that segments across buffer boundaries, for terminal input paths
• Configurable UTF-8 error handling and POSIX-compatible control-character semantics
• Zero dependencies; no allocation in the core measurement path

Installation

Add the dependency to shard.yml:

dependencies:
  wcwidth:
    github: shpeckman/wcwidth-cr

Then run:

shards install

And require it:

require "wcwidth"

Usage

Whole-string width

WCWidth.swidth("文本")        # => 4
WCWidth.swidth("👨‍👩‍👧‍👦")       # => 2
WCWidth.width('文')           # => 2  (single code point)

Column helpers

WCWidth.grapheme_count("🇯🇵🇩🇪")            # => 2
WCWidth.truncate("文文文", 5)                # => "文文"
WCWidth.truncate("Hello world", 8, "…")     # => "Hello w…"
WCWidth.ljust("文", 6)                      # => "文    "

truncate never splits a cluster. A wide cluster that would straddle the limit is dropped rather than half-rendered, so truncate("文文文", 5) yields two columns of slack rather than a broken glyph.

One-shot measurement

measure returns the width of the first grapheme and how far it extends:

m = WCWidth.measure("e\u0301x")
m.width      # => 1
m.consumed   # => 3  (bytes)

The same works over code points, where consumed counts code points instead:

WCWidth.swidth(Slice[0x41_u32, 0x6587_u32])   # => 3

m = WCWidth.measure(Slice[0x65_u32, 0x301_u32, 0x78_u32])
m.width      # => 1
m.consumed   # => 2  (stops before 'x')

Grapheme iteration

WCWidth.each_grapheme("A文") { |g, w| puts "#{g}=#{w}" }
# A=1
# 文=2

WCWidth.graphemes("A文😀").map { |_, w| w }.to_a   # => [1, 2, 2]
WCWidth.grapheme_next("e\u0301x")                  # => 3

graphemes returns a lazy Iterator({String, Int32}), so it composes with map, sum, select, and friends.

Streaming

Drive the state machine directly to segment across reads. A persistent State carries the context needed to decide breaks that span buffer boundaries:

st = WCWidth::State.new
cl = WCWidth::Cluster.new
total = 0
started = false

each_codepoint_from_terminal do |cp|
  if st.grapheme_break?(cp) && started
    total += cl.width
    cl.reset
  end
  cl.push(cp)
  started = true
end

total += cl.width if started

API Reference

Measurement

Iteration

Layout

Streaming types

Policies

UTF8Policy controls malformed input:

• Replace (default) — malformed, truncated, overlong, or surrogate sequences become U+FFFD, advancing one byte. Never reads past the given length.
• Strict — stop at the first malformed sequence and report what was consumed.

CtrlPolicy controls control characters in swidth:

• Skip (default) — controls contribute 0; the result is the sum of printable clusters.
• Fail — any control makes the whole call return -1, matching POSIX wcswidth.

WCWidth.swidth("A\u001BB", WCWidth::UTF8Policy::Replace, WCWidth::CtrlPolicy::Fail)  # => -1
WCWidth.swidth(Bytes[0x41, 0x80], WCWidth::UTF8Policy::Strict)                       # => 1

Introspection

WCWidth.unicode_version    # => "17.0.0"
WCWidth::ACTIVE_WIDTH_CAP  # => 2

Configuration

Cluster-width cap

By default a cluster is capped at two columns, matching the Mode 2027 majority (WezTerm, Ghostty, foot). To report the base character's own width instead, without a cap (kitty-style), compile with:

crystal build -Dwcw_cluster_width_cap_0 app.cr

The flag must be set consistently for the library and any code reading ACTIVE_WIDTH_CAP.

Regenerating tables

src/wcwidth/tables.cr is generated. To rebuild it from the upstream C header:

make tables

Behavior Notes

• Controls report -1 per code point and per cluster; swidth applies CtrlPolicy to decide what that means for the whole string.
• Under UTF8Policy::Replace the UTF-8 entry points always advance at least one byte, so iteration cannot stall on malformed input.
• Regional indicators pair up: two form one two-column cluster, and a third begins a new one.
• All functions are reentrant. State lives only in caller-owned State and Cluster values; the tables are read-only and no globals are mutated.

Verification

The port is checked against the original C library rather than only against hand-written expectations:

• All 1,114,112 code point widths are byte-identical to the C implementation
• All 766 official Unicode GraphemeBreakTest cases pass
• 60,000 fuzz cases covering malformed UTF-8, overlong encodings, surrogates, and random bytes produce identical results across all nine public functions
• 200,000 randomized strings satisfy the layout invariants, including that truncate is maximal — no additional cluster could fit
• 60 specs

make spec

Performance

Release build, measured against the same corpora as the C benchmark (make bench):

Throughput is comparable to the gcc -O2 build of the original.

License

MIT, matching upstream. See LICENSE.

The lookup tables and spec/graphemebreaktest.txt are derived from the Unicode Character Database and are subject to the Unicode License v3; see LICENSE-UNICODE.