# October 20 inference benchmark: recovered evidence

Reviewed October 22, 2026. These are historical published observations, not newly
reproduced experiments. The complete original clients were recovered unchanged;
no reconstructed benchmark harness is substituted for them.

Files (download into one directory, with the JSONL and article in its parent):

- [llama_server.rb](llama_server.rb): original llama.cpp HTTP load client.
- [ollama.rb](ollama.rb): original Ollama HTTP load client.
- [manifest.json](manifest.json): exact prompt, known request/launch settings,
  version claims, model blob identifier with provenance, unknowns and original hashes.
- [check.rb](check.rb): offline artifact, run matrix, arithmetic and article check.
- [test.rb](test.rb) and [fixture.rb](fixture.rb): Minitest and deterministic offline HTTP fixture.
- [Historical results](../inference-roadmap-bench.jsonl): all 26 rows, unchanged.
- [Article](../inference-roadmap.html): corrected fixed-16 comparison.

## Safe checks (no model, service, network, or GPU required)

Use Ruby with Minitest available. No dependency installation is needed on the source
host (tested with Ruby 3.3.8). From this directory:

```sh
ruby check.rb
ruby test.rb
```

The fixture runs both complete original clients as subprocesses with Net::HTTP
replaced before loading them. It asserts endpoint, options, exact prompts, warmup,
token totals and timing arithmetic. Its numbers are fabricated test inputs, not
benchmark observations. Neither test command contacts the hard-coded endpoints.

## Historical client invocations

Do not run the original clients just to check the bundle: even with no arguments
they send a warmup request. They use fixed loopback ports, have no HTTP status/error
validation, and can contend with other workloads. Reserve a separate idle machine
and confirm ownership of both ports and GPU before any future experiment.

The recovered commands used the following client sequences (one invocation per
server slot setting, SLOTS = 1, 4, 8):

```sh
SLOTS=1 ruby llama_server.rb 1 4 8 16  # GPU; repeat with SLOTS=4 and 8 on matching servers
SLOTS=1 ruby llama_server.rb 1 4 8     # historical CPU fallback sequence
ruby ollama.rb 1 1 2 4 8
```

SLOTS only labels the output. It does not configure the server. Each argument is a
burst of that many total requests, not a sustained load. Each invocation first
warms with `hi`; llama uses a numbered prompt, Ollama repeats the same prompt.
The exact trailing spaces and all request options are in the original scripts and
manifest. Ollama caching was not disabled; the former article's blanket cache claim
was incorrect. The first numbered GPU prompt was logged as 70 tokens, not 80.

## Recovered llama server launch configuration

This is documentation, not a service-management script. Paths are portable placeholders;
no process is started or stopped by the safe checks. The original explicit invocation was:

```sh
"$SERVER" -m "$MODEL" -ngl 99 -np "$SLOTS" -c "$((1024 * SLOTS))" --host 127.0.0.1 --port 8195 --metrics
```

For GPU runs, the recorded command also set
`LD_LIBRARY_PATH="$D/cuda_v13:$D"` and
`GGML_BACKEND_PATH="$D/cuda_v13/libggml-cuda.so"`, where D was the Ollama library
directory. CPU-labelled runs used the same arguments without those explicit
environment overrides and fell back to CPU. The historical launcher waited for
`/health` before invoking the client. No automatic launch/kill loop is included.

The manifest records the model blob's SHA-256-shaped filename identifier from the
historical command and log. It has **not** been independently rehashed. Full inherited
environment, binary/library hashes, engine defaults and Ollama service configuration
are not completely preserved. These missing inputs prevent a claim of fully exact
reproduction even though the complete clients and explicit server arguments survived.
Record those inputs, actual model/binary hashes, hardware, request responses, failures,
and repeated trials in a new manifest for any future run. Save new results separately.

## What the data supports

For GPU runs with 16 total requests and 3,200 generated tokens, aggregate throughput
is 30.3, 64.9 and 80.7 tokens/s at 1, 4 and 8 slots. Slowest-request times are 105.50,
49.32 and 39.63 seconds. Eight slots deliver about 2.66 times the aggregate throughput,
not faster isolated responses. The single-request rows are 6.50, 6.65 and 6.62 seconds.
Backend decode rates are not end-to-end latency or time to first token.

The checker verifies the immutable original hashes, complete ordered run matrix,
200 generated tokens per request, rate-array sizes, rounded throughput arithmetic
and the article's fixed-16 table. This verifies internal consistency, not historical
measurement accuracy, output quality, GPU memory attribution or reproducibility.
Memory figures and versions remain historical reports; no new inference experiment,
TTFT measurement, service tuning or large benchmark was run for this repair.