autoresearch.cr
autoresearch.cr
A Crystal port of Karpathy's autoresearch, on tensomite instead of Python and PyTorch.
In autoresearch an agent edits one training script, trains a small language model for a fixed five minutes, and keeps the edit only if bits per byte on a held-out set fell. Think of a chef allowed to change one recipe card, judged only on how the dish tastes after a fixed time on the stove. This port keeps the card, the stove timer and the tasting exactly as in the original; only the kitchen (Crystal and tensomite) changes. It is also a proving ground for tensomite: every place the recipe strains the library becomes an issue there.
See SCOPE.md for the goal and requirements, PLAN.md for the milestones, DESIGN.md for how it is put together, and DEVELOPMENT.md for how to work on it.
Status
Milestone M0 (foundation). There is no train or prepare yet. What builds and runs is the spike, spikes/gpt_spike.cr: a 4-layer, 0.9 M-parameter model with the original's structure and a MuonAdamW optimizer, trained on a synthetic token stream. It shows the pieces run on tensomite; later milestones check them against the original.
Quick start
- Install Crystal 1.22 or later. On Linux, also install OpenBLAS:
sudo apt-get install libopenblas-dev - Install the shards and build the linter:
make deps - Run the specs and the linter:
make test - Build the spike (a release build, about 30 s):
make build - Train it for 200 steps:
It prints the parameter count, the loss every 20 steps (from about 5.54 down to below 0.01) and the time taken.bin/gpt_spike 200bin/gpt_spike 200 accumtakes the same steps with gradient accumulation over two micro-batches, and should print the same losses to the shown precision.
Contributions
By invitation only for the time being.
autoresearch.cr
- 0
- 0
- 0
- 0
- 2
- 29 minutes ago
- November 21, 2026
Mozilla Public License 2.0
Sat, 21 Nov 2026 23:43:08 GMT