mothergod

What this is

Compression runs in three stages. First, reversible transforms rearrange bytes to expose patterns a plain byte-by-byte view hides. Second, an LZ stage finds repeated stretches of the input and works out the cheapest way to describe them, including cheap references back to a handful of recently used distances. Third, an arithmetic coder encodes what remains, blending predictions from several models and adjusting that blend as it reads. Ratio came first, so judge it on bits per byte; the speed section below says what that order cost. Every design decision traces to a recorded experiment in the research journal, rejections included.

Measured

Aggregate bits per byte on the two named corpora, lower is better. Canterbury is 11 files and 2.8 MB, Silesia is 12 files and 212 MB; both are pinned by URL and SHA-256, fetched at measurement time and never committed. Measured 2026-10-18 against gzip 1.12, Zstandard 1.5.7 and XZ Utils 5.4.5, at the flags in the header row.

Read it honestly: mothergod beats both zstd -19 and xz -9e in aggregate on Canterbury, and loses to both in aggregate on Silesia. Per file, against whichever of the two is stronger on that file, it wins 7 of 11 on Canterbury and 1 of 12 on Silesia. Closing Silesia is the current milestone. The per-file tables, the throughput columns, and the one command that regenerates each report are in docs/benchmarks/canterbury.md and silesia.md.

Speed

Slow, and that is the honest headline. mothergod's codec has no internal parallelism; the reports time it one file at a time, with nothing else running, on one CI machine (AMD EPYC 9V45, 4 logical cores), so each rate is comparable between files.

Silesia's 212 MB took about nineteen minutes to compress and about one minute to read back. Encoding carries the optimal parse, so it runs roughly eighteen times slower than decoding.

One caveat the reports state themselves: these are single-run figures from one machine, not a cross-machine claim. One more worth adding: an aggregate is total bytes over total time, so it is not a typical file. Per-file decode rates in those reports run from 0.9 to 35.3 MB/s. Speed is measured on every benchmark run and becomes a target of its own at the speed-tiers milestone on the roadmap; nothing before then is tuned for it.

Try it

There is no download: no release, no tag, no package. Building it yourself is the only way to run it, and the core crate has zero runtime dependencies, so this compiles exactly one crate.

git clone https://github.com/bugabinga/mothergod
cd mothergod
cargo build --release --bin mothergod

The first run needs no file of yours, because the clone ships one:

$ ./target/release/mothergod -v -c LICENSE > /dev/null
mothergod: LICENSE: 1074 -> 630 bytes (58.6%)

-v asked for that line; without it, a run that worked prints nothing. -c sent the compressed bytes to stdout, which /dev/null swallowed, so the run writes no file and repeats cleanly. One 1 KB file is a smoke test, not a benchmark; the tables above are the benchmark. Then round-trip something of yours:

./target/release/mothergod    < FILE      > FILE.mgdc
./target/release/mothergod -d < FILE.mgdc > FILE.out
cmp FILE FILE.out

The CLI speaks the grammar gzip, xz and zstd already taught you: it compresses, -d decompresses, stdin to stdout when you name no file. That last line is the point of running it at all, and cmp printing nothing is it passing. Given a file, it writes <file>.mgdc beside it, and unlike gzip it never deletes the input. --help lists the rest on one screen: flags you know from gzip, and no compression levels or tuning knobs. --version also prints the format version the build reads; put it in a bug report. Expect the encode to take a while, at the rates above.

Who builds it

mothergod began as an autoresearch loop. On 2026-08-19 one Claude session ran about 40 iterations of propose, score bits per byte on validation data, keep or reject, record; a proposed filter had to invert first. Bits per byte is also the metric karpathy/autoresearch minimizes, on language models instead of files. That harness is preserved at commit 1a3b1c8, and its findings open the research journal. The loop still runs; around it grew a versioned format, a decoder fuzzed against hostile input, and review by an agent that did not write the change.

Day-to-day development is done by Claude agents running on GitHub Actions: triage, implementation, adversarial code review, research, releases. Slowly, in public, like a real team would. A human operator holds the veto and the keys. That is the second experiment in this repository, and the table above is the first one's report card. How it works: agents/GOVERNANCE.md, and the seats are listed on the agents page. What it has decided, and what it later reversed, is on the decisions page.

Principles

• Lossless is sacred: the decoder never panics on any input, adversarial or not. Backed by 4 fuzz targets, including decode_arbitrary, plus an adversarial suite and a torture suite that feed it truncated and bit-flipped input on purpose. Every frame ends in a CRC-32 of its own bytes, so a file damaged on disk or in transit by a flipped bit fails with an error, never decodes to output that looks fine and is not. Cumulative fuzz time and the crash count run live on the status page.
• Every benchmark claim names its corpus; a ratio without a corpus is not a claim. bench/corpus.toml pins the 13 archives behind every number above by URL and SHA-256, per the corpus policy.
• Every experiment, accepted or rejected, is recorded: rejections are knowledge too. research/progress.jsonl holds at least 90 entries, at least 10 of them rejected, each with its own hypothesis and mechanism of failure.
• Verification is independent of the proposer: agents never grade their own work. CLAUDE.md's hard rules 3 and 8 forbid an agent merging its own pull request or grading its own claim; agents/GOVERNANCE.md names the reviewer that enforces both.

Follow along