c0-cr

Super fast structured serialization

C0DATA

A data system built on ASCII C0 control codes. One vocabulary, multiple shapes -- tabular data, hierarchical documents, key-value config, diffs, logs, and more.

C0DATA sits between human-readable text formats (JSON, YAML, TOML) and opaque binary formats (protobuf, msgpack). Values are plain UTF-8 text. Structure is expressed through single-byte control codes -- compact, zero-copy friendly, and inspectable with minimal tooling.

This is the Crystal reference implementation of the 2026-10 (pre-1.0) redesign, specified in c0-spec.

The Idea

ASCII has 32 control codes (0x00-0x1F) that were designed in the 1960s for structuring data transmissions. Four of them -- FS, GS, RS, US -- are literal data separators at four hierarchical levels. C0DATA takes them, and a dozen of their neighbours, at their word.

␖␜mydb
  ␝users
    ␁␞name␟amount␟type
    ␞Alice␟1502.30␟DEPOSIT
    ␞Bob␟340.00␟WITHDRAWAL
  ␝products
    ␁␞id␟product␟qty
    ␞01␟Widget␟100
    ␞02␟Gadget␟250
␄

That's a complete document with two tables, header rows, and typed rows. Each glyph (␖ ␜ ␝ ␁ ␞ ␟ ␄) is a single-byte control code. No braces, no commas; quoting only where a value holds whitespace or a control code. Whitespace touching a control code is layout -- the indentation is just for readability.

Control Codes

Byte Abbr Glyph Role
0x01 SOH ␁ A key for the value after it; ␁␞ a header row
0x02 STX ␂ Start of text (a literal value)
0x03 ETX ␃ End of text
0x04 EOT ␄ End of document
0x05 ENQ ␅ Reference
0x07 BEL ␇ Comment: a note, or marks the next element dead
0x10 DLE ␐ Recurse: open a nested level (escape inside text)
0x16 SYN ␖ Start of document, then the heading
0x17 ETB ␗ Commit marker (stream mode)
0x18 CAN ␘ Close a nested level
0x19 EM ␙ End of medium: a raw blob follows
0x1A SUB ␚ Subordinate: the element after is a child of the element before (GS: subsection, US: sub-unit)
0x1C FS ␜ File separator
0x1D GS ␝ Group / table / section separator
0x1E RS ␞ Record / row separator
0x1F US ␟ Unit / field separator

Data Shapes

Tabular

␝users
  ␁␞name␟amount␟type
  ␞Alice␟1502.30␟DEPOSIT

Key-value (a keyed record is a map)

␝database
  ␞␁host␟localhost␁port␟5432

Nested values

A field can hold a whole level: a list, a table, a map, even a document.

␞Alice␟␐␞Admin␟Editor␘␟␐␞␁city␟Springfield␁street␟1 Main␘

Text

A value with edge whitespace, a line break, a tab or any control code is written as text; inside, only seven codes need a ␐ escape.

␞note␟␂line one
line two␃

Document

␖␜My Document
␝Chapter 1
␞First paragraph.
␚␝Section 1.1
␞Nested content.
␄

Stream (append-only logs)

ETB commits each appended block; a crashed append never writes its commit, so replay skips the torn tail.

␖␁shape␟stream␁␞op␟id␗
␞create␟a1b2␗
␞name␟draft-2␗

Diff

Records are anchored patterns; old␚␟new inside one (the sub-unit of old) is the substitution. See DIFF.md in the spec.

␖␁shape␟diff
␜greeting.txt
␞␂Hello ␃␟world␚␟universe␟!
␄

Performance

The scanner's hot loop is a single comparison: byte < 0x20. Single-byte delimiters, zero-copy reads, SIMD-friendly.

Installation

dependencies:
  c0:
    github: c0data/c0-cr

Usage

require "c0"

Building

buf = C0::Builder.build do |b|
  b.syn(shape: "tabular")
  b.file("mydb")
  b.group("users", ["name", "amount", "roles"]) do
    b.record
    b.field("Alice")
    b.field("1502.30")
    b.list(["Admin", "Editor"])          # ␐␞Admin␟Editor␘
  end
  b.group("config")
  b.map({"port" => "5432", "host" => "localhost"})   # order kept
  b.eot
end

Values are written in their one canonical spelling: plain when they can be, text otherwise. b.level { ... } opens a nested level as the next unit; b.key(name, value) adds a keyed unit; b.note(text) a comment.

Underneath, the builder has one method per code: fs, gs, rs, us, sub, soh, dle, can, bel, enq, syn, eot, etb, em. Each writes its code and, when given, what follows it, so a chain reads like the bytes, and anything the grammar allows can be written:

b.rs("Alice").us("1502.30")                               # ␞Alice␟1502.30
b.rs.soh("host").us("localhost").soh("port").us("5432")   # ␞␁host␟localhost␁port␟5432
b.gs("users").soh.rs("name").us("amount")                 # header row
b.bel.gs("old")                                           # a dead group
b.rs("old").sub.us("new")                                 # ␞old␚␟new  (new is a sub-unit of old)
b.gs("intro", depth: 2)                                   # ␚␝intro   (a subsection)
b.rs.enq("tags").us.enq { b.gs("tags").rs("001") }        # ␞␅tags␟␅␐␝tags␞001␘

The composed methods above are built on these and cannot write ill-formed output; with the primitives, well-formedness is yours. The two layers share state and mix freely.

Reading

doc = C0::Document.new(buf)
doc.heading.shape                 # => "tabular"
doc.name                          # => "mydb"
users = doc["users"]
users.headers                     # => ["name", "amount", "roles"]
rec = users.record(0)
rec.value(0)                      # => "Alice" (zero-copy slice)
rec.list(2)                       # => ["Admin", "Editor"]
rec.level(2).records              # the same level, as a body
doc["config"].record(0)["host"]   # => "localhost"
doc.warnings                      # missing SYN/EOT, unknown heading keys

C0::Container.new(buf) reads a file holding several documents and an EM blob tail (container.blob("name")). C0.validate(bytes) checks well-formedness against the grammar; C0.canonical?(bytes) checks the canonical constraints.

The model is the result: a dead element (␇␞…) is parsed and dropped with everything it holds. SUB makes the element after it a child of the element before, at that code's grain: ␚␝s is a subsection at depth two, ␞a␟b␚␟c is two units with c a sub-unit of b (not a unit of the record); ␚␞, ␚␜ and ␚␁ are reserved and refused. rec.unit(n) and rec.unit(key) give a unit with its subs. A reference (rec.ref(i)) is a group name or a pattern, the level ␅␐␝tags␞001␟label␘ parsed as a body; ref.path linearises it.

Pretty form

puts C0::Pretty.format(buf)                       # one element per line
puts C0::Pretty.format(buf, mode: C0::Pretty::FormatMode::Spaced)
C0::Pretty.parse(pretty) == buf                  # round trip

Serializable

class User
  include C0::Serializable
  property name : String
  property roles : Array(String)        # a list level
  @[C0::Field(key: "amt")]
  property amount : Float64
  def initialize(@name, @roles, @amount); end
end

User.new("Alice", ["admin"], 1.5).to_c0   # ␝user␁␞name␟roles␟amt␞Alice␟␐␞admin␘␟1.5
User.from_c0(bytes)
[u1, u2].to_c0
{"b" => "2", "a" => "1"}.to_c0("cfg")      # ␝cfg␞␁a␟1␁b␟2

JSON / YAML / CSV

C0::JSON.to_json(buf)        # positional record → array, keyed → object,
C0::JSON.from_json(json)     # header row → array of objects; nothing guessed
C0::JSON.to_yaml(buf) / C0::JSON.from_yaml(yaml)
C0::CSV.from_csv(csv, "users") / C0::CSV.to_csv(buf)

C0DIFF

diff = C0::Diff.build do |d|
  d.file("greeting.txt") do
    d.record { |r| r.anchor("Hello "); r.sub("world", "universe"); r.anchor("!") }
    d.insert_after("universe!", "\n")
    d.assert("Hello")                               # must be present once; nothing changes
  end
end
C0::Diff.apply(diff, {"greeting.txt" => source})   # validates everything first
C0::Diff.apply_files(diff, "src")

Each record's search text must match once from the previous match to the end of the file. There is no replace-all mark: a tool expands one into anchored records against the target (DIFF.md). Anything the text diff does not use (keyed units, levels, references, header rows, GS, dead records, two sub-units on one unit) is refused.

Stream logs

C0::Stream::Writer.open("events.c0") do |w|
  w.start                              # SYN heading, committed
  w.header("op", "id")
  w.record("create", "a1")
  w.batch { |b| b.record("a", "1"); b.record("b", "2") }   # one commit
end

r = C0::Stream::Reader.read("events.c0")
r.torn?            # uncommitted tail?
r.damaged          # indexes of committed blocks that are not well-formed
r.each_record { |rec| ... }

Writer.open repairs a torn tail before appending. A damaged block delivers nothing; by default the reader yields the records before it and raises C0::Stream::DamagedBlockError with the offset to resume at, and with Reader.new(buf, continue_past_damage: true) it reads on past it.

Conformance

The shared vectors live in the c0-spec submodule at c0-spec/ (git submodule update --init after cloning); crystal spec runs them along with the unit specs.

c0fmt

The command-line tool lives in its own repository: https://github.com/c0data/c0-fmt.

License

MIT

Repository

c0-cr

Owner
Statistic
  • 14
  • 0
  • 1
  • 2
  • 0
  • 3 days ago
  • March 5, 2026
License

MIT License

Links
Synced at

Sun, 15 Nov 2026 16:34:58 GMT

Languages