gota・滴

What a micro-benchmark is

A micro-benchmark isolates one small operation — a hash, a parse, a buffer copy — and runs it in a tight loop to measure how fast that single code path goes. It deliberately leaves out everything around it: no network, no disk, no real workload. That isolation is the whole point. It is what makes one operation's speed a clean, comparable number — here, the same operation across eight languages.

These answer different questions and are complements, not rivals: a micro-benchmark tells you which implementation of an operation is faster, a macro benchmark tells you whether the whole system is fast enough, and a profiler tells you where to look. gota lives in the first row.

gota sits in that first row alongside criterion, JMH, and Google Benchmark, but it is a different kind of thing. Each of those is one language and a dependency you install — criterion in Rust, JMH in Java, Google Benchmark in C++. gota is the opposite: one ~20-line protocol you copy, not a package you depend on, spanning all eight languages so their results land in the same table. The numbers compare across languages, which a single-language harness cannot do.

Three layers, one contract

A runner is usable on its own; the orchestrator is just the conductor that runs them together; the report is a pure consumer of the JSON. You can stop at any layer. Between the first two sits the protocol every runner obeys.

Eight languages, one protocol

Two files per language: gota.* is the harness you copy untouched; runner.* is your code, where one seam (run(impl, register) then bench(name, op)) plugs an operation into the timing loop. Each runner is independently runnable.

Python is tinted because it also hosts the orchestrator (harness.py + your run.py). Every other row is just a runner. Full build notes live in each templates/<lang>/README.md.

See a live report

The bundled example — FNV-1a across all eight languages — rendered by report.py into a standalone, sortable HTML viewer. One toggle in the report flips the same measurement between byte throughput (MB/s) and operations per second (one op = hashing the whole 64 KiB buffer): same numbers, same ranking, two units.

A self-contained page with a file picker, so you can drop any other results.json in and re-render. Generated from the committed example data, never hand-edited.

The protocol is the product

Every runner obeys the same recipe, so numbers compare even though each runner is native to its language. This is the same discipline a multi-port project uses to keep ports in sync; here the ports are the timing harnesses.

• Same three CLI args — buffer bytes, warmup seconds, measure seconds — passed identically to every runner.
• Peak of batches, with a stability check. Warm up the JIT, grow a batch until it runs ≥ 100ms, then report the fastest batch's MB/s next to the median of all batches — a close gap means a clean run, a wide one means the peak is noisy. The clock is read only at batch boundaries, never around one tiny op.
• One JSON line per benchmark on stdout ({"impl","bench","mbps","mbps_median","iters"}) — the entire wire format between a runner and the orchestrator.
• Kept equivalent across all eight templates. Change the loop in one and you change it in all of them, and in PROTOCOL.md, or they stop being comparable.

The easiest way to measure nothing

The classic micro-benchmark trap: if nothing observes your operation's result, an optimizing compiler is free to delete the whole computation — and you measure nanoseconds of nothing, a confident wrong number. gota's templates guard against it in every language.

Honest by design

gota reports peak (unimpeded) throughput on a normal machine, which is right for comparing implementations and optimistic for real-world latency. It says so, and it never invents a number.

It is also deliberately not some things: not a nanosecond-latency tool (use criterion or JMH for that), and not lab-grade absolute numbers (for those you would pin cores and lock frequency on a dedicated box). It is a comparable picture, drop by drop.

Copy and run

Copy harness.py and the templates/<lang> for each language you benchmark, swap your operation into each runner.*, write a small run.py, and run it. The bundled example ties four languages together end to end.

# run one language's runner standalone (prints one JSON line)
python3 templates/python/runner.py 65536 0.2 0.4

# run the full example: build + run every runner, then tabulate
python3 examples/run.py            # writes results.json + RESULTS.md
python3 report.py examples/results.json -o examples/report.html