jawas

Swiss-army backup tool: MySQL/MariaDB, PostgreSQL and SQLite dumps with rotation and compression

= jawas :toc: left

A Swiss-army backup tool written in Crystal: MySQL/MariaDB, PostgreSQL and SQLite dumps, daily/weekly/monthly rotation, a choice of compressor, SHA-256 checksums, TOML configuration.

jawas builds on the ideas of https://sourceforge.net/projects/automysqlbackup/[automysqlbackup] (daily/weekly/monthly rotation, one archive per database, pre/post hooks) and widens them: several engines, several servers in one configuration, public-key age encryption, backup verification and restore, and a roadmap towards remote upload.

Version française : link:README.fr.adoc[README.fr.adoc].

== Features (0.3.0)

  • Engines: MySQL and MariaDB (mysqldump / mariadb-dump), PostgreSQL (pg_dump, plain SQL or custom format, plus roles via pg_dumpall --globals-only), SQLite (consistent hot copy through .backup, or SQL dump).
  • Several sources per configuration ([[source]]), automatic database discovery, pattern (test_*) and table exclusions.
  • Rotation: every run feeds daily; on the chosen day the same file is hard-linked (zero extra bytes) into weekly and/or monthly. Per-period retention in days and a keep_min floor so a long outage can never empty the storage.
  • Streaming compression: none, gzip, pigz, bzip2, xz, zstd, with level and threads.
  • age encryption (0.2): after compression, to one or several public keys (age1…, ssh-ed25519, ssh-rsa) inline or in an age -R file. The server holds no private key: it cannot read its own backups back.
  • Verification and restore (0.3): verify checks checksum, archive and content; restore loads a backup back, into a throw-away database by default.
  • Integrity: write to .part then atomic rename, .sha256 sidecar (sha256sum -c format), 0600 files and 0700 directories, latest/ link to the newest dump.
  • Credential hygiene: no clear-text password in the configuration nor on the command line (password_env, password_file, defaults_file).
  • Operations: lock against concurrent runs, pre/post hooks, last-run.json JSON report, monitoring-friendly exit codes.

== Installation

[source,shell]

git clone git@github.com-aloli:aloli-crystal/jawas.git shards build --release --production ln -s "$PWD/bin/jawas" ~/bin/jawas

External tools, depending on the engines and compressor in use: mysqldump/mariadb-dump, pg_dump/pg_dumpall/psql, sqlite3, gzip/pigz/bzip2/xz/zstd, and age (or rage) for encryption.

== Quick start

[source,shell]

jawas sample > ~/.config/jawas/config.toml jawas check jawas backup jawas backup --apply jawas list

backup and prune are dry-run by default: they show what they would do. --apply (-a) really runs them. Cron example:

[source]

15 3 * * * /usr/local/bin/jawas backup --apply --quiet

== Configuration

Lookup order: --config PATH, ${XDG_CONFIG_HOME:-~/.config}/jawas/config.toml (skipped with --no-user-config), /etc/jawas/config.toml, /usr/local/etc/jawas/config.toml. jawas sample prints a complete, commented template. Unknown keys are rejected (no silent typo).

[source,toml]

[storage] dir = "/var/backups/jawas"

[schedule] weekly_day = 6 # Saturday (1 = Monday … 7 = Sunday) monthly_day = 1

[retention] # days daily = 7 weekly = 35 monthly = 180 keep_min = 2

[compression] tool = "zstd" level = 6

[encryption] recipients = ["age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p"]

[[source]] name = "mariadb" engine = "mariadb" socket = "/run/mysqld/mysqld.sock" user = "backup" password_file = "/etc/jawas/mariadb.password"

[[source]] name = "pg" engine = "postgresql" socket = "/var/run/postgresql" user = "postgres"

== Encryption and reading back

The [encryption] section turns encryption on; without it, backups stay in clear. Files get the .age extension after the compressor's (app_….daily.sql.zst.age) and the .sha256 sum covers the encrypted file. jawas check makes sure age accepts every recipient.

[source,toml]

[encryption] recipients = ["age1…", "ssh-ed25519 AAAA… ops@workstation"] recipients_file = "/usr/local/etc/jawas/recipients.txt" # one key per line

binary = "rage"


Create the key pair on the operator's workstation (never on the server), then read a backup back:

[source,shell]

age-keygen -o ~/.config/age/jawas.txt age -d -i ~/.config/age/jawas.txt app_2026-11-16_03h15.daily.sql.zst.age | zstd -dc > app.sql

Keep the private key safe (with a backup copy): without it, encrypted backups are lost.

== Verification and restore

jawas verify checks, by default, the latest backup of each database (-A: all of them), on three levels:

  1. checksum: the file matches its .sha256;
  2. archive: it decrypts (with -i) and decompresses to the end;
  3. content: the dump is complete (mysqldump or pg_dump end marker, PGDMP header, final COMMIT; or PRAGMA integrity_check for SQLite).

Without -i, an encrypted backup is only checked on its checksum ("partial"). The exit code is 2 when a check fails: jawas verify -q fits a scheduled job.

[source,shell]

jawas verify jawas verify -A -i ~/.config/age/jawas.txt

jawas restore loads a backup back. Without --apply, it shows the chosen backup and the target. By default:

  • the backup is the latest of the database (-D YYYY-MM-DD: the latest of that day; -f: a given file);
  • the target is a throw-away database <db>_r<YYYYMMDDHHMM> on the same server (SQLite: sibling file <db>.r<YYYYMMDDHHMM>.sqlite3), to inspect then drop.

[source,shell]

jawas restore -d app jawas restore -d app --apply jawas restore -s pg -d app -D 2026-11-14 -t app_yesterday --apply jawas restore -d app -t app -R -i ~/.config/age/jawas.txt --apply

Safeguards:

  • the SHA-256 checksum is checked before anything is loaded;
  • an existing target is only overwritten with -R/--replace; overwriting the original database thus takes -t <db> -R --apply. For MySQL and PostgreSQL the database is dropped then recreated: restore into a throw-away database first to check. SQLite is replaced in one go (file renamed after PRAGMA integrity_check);
  • a throw-away database created for a failed restore is dropped;
  • _globals (PostgreSQL roles) is replayed by hand: decode the file, then psql -d postgres -f -.

Loading goes through mysql/mariadb (CREATE DATABASE and USE header removed to aim at the target), psql -v ON_ERROR_STOP=1 or pg_restore --exit-on-error, and sqlite3.

== Storage layout

[source]

/var/backups/jawas/ ├── mariadb/ │ ├── daily/app/app_2026-11-16_03h15.daily.sql.zst │ ├── daily/app/app_2026-11-16_03h15.daily.sql.zst.sha256 │ ├── weekly/app/… (hard links) │ ├── monthly/app/… │ └── latest/app.sql.zst → ../daily/app/app_2026-11-16_03h15.daily.sql.zst ├── pg/… └── last-run.json

With [encryption], names end in .age (app_2026-11-16_03h15.daily.sql.zst.age, latest/app.sql.zst.age).

== Commands

[cols="1,1,4"] |=== | Command | Short | Purpose

| backup | b | Dumps, links, rotation, report (dry-run without --apply) | prune | p | Rotation only (dry-run without --apply) | list | l | Inventory of the stored backups | verify | v | Checksum, archive and content checks | restore | r | Restore, throw-away database by default (dry-run without --apply) | check | c | Configuration, executables, storage, connections | sample | s | Configuration template | help | h | Global help or help <command> |===

Options: -c/--config, -U/--no-user-config, -a/--apply, -s/--source and -d/--database (repeatable or a,b), -P/--no-prune, -j/--json, -q/--quiet, -v/--version, -h/--help; for verify and restore: -i/--identity (age private key), -A/--all, -D/--date, -f/--file, -t/--into, -R/--replace.

Exit codes: 0 success, 1 usage or configuration error, 2 at least one database failed, 3 lock held by another run.

== Differences from automysqlbackup

  • Validated TOML configuration instead of a sourced shell file.
  • One run = a continuous daily series; weekly/monthly are hard links instead of separate dumps.
  • Retention with a keep_min floor.
  • No password in the configuration (temporary 0600 option file for MySQL, PGPASSWORD for PostgreSQL).
  • Public-key age encryption instead of shared-password openssl.
  • No mail report nor differential mode yet: see the roadmap.

== Roadmap

  1. 0.2 — age encryption (multiple recipients): shipped.
  2. 0.3 — verify and restore: shipped.
  3. 0.4 — remote upload (SFTP/rsync, S3 object storage: OVH, Scaleway…), mail/webhook report notification.
  4. 0.5 — fine-grained GFS retention, monitoring metrics.
  5. Later — "file" sources (rsync, tar archives), beyond databases.

== Development

[source,shell]

shards install crystal spec crystal tool format src/ spec/ crystal run lib/ameba/bin/ameba.cr -- src spec

== License

MIT — see LICENSE.

Repository

jawas

Owner
Statistic
  • 0
  • 0
  • 0
  • 0
  • 2
  • about 3 hours ago
  • November 16, 2026
License

MIT License

Links
Synced at

Mon, 16 Nov 2026 12:22:44 GMT

Languages