Your local model starts answering slowly. Before changing cache settings or adding parallel slots, measure when the client actually receives answer text—not just when the connection opens.

This guide includes a small, dependency-free Python probe. It reports time to first nonempty content and time until the stream completes. It does not pretend those numbers are the model engine’s internal time to first token (TTFT).

Verification boundary: the probe was tested against a controlled local SSE server, including delayed content, empty responses, and truncated streams. This is not a hardware benchmark or a claim that a particular llama.cpp build/model was tested. Official protocol documentation was checked on October 10, 2026; verify compatibility with your installed server.

1. Download and inspect the probe

You need Python 3 and an already-running llama-server with a model that supports chat completion. The official llama.cpp server documentation describes the streaming /v1/chat/completions endpoint used here.

Download measure-llama-stream.py and inspect it before running:

curl -fL https://7minai.com/downloads/measure-llama-stream.py \
  -o measure-llama-stream.py
less measure-llama-stream.py

The file uses only Python’s standard library. It sends your chosen prompt to your chosen endpoint; it does not send results to 7minAI. Its normal output contains measurements, not the generated response text.

Keep your server bound to localhost or a protected private service. Do not expose an inference port publicly just to run this test. If your server requires authentication, the script reads LLAMA_API_KEY from your environment; it does not require putting a key in the command line.

2. Run three sequential requests

Replace YOUR_SERVED_MODEL_ID with the identifier accepted by your server:

python3 measure-llama-stream.py \
  --url http://127.0.0.1:8080/v1/chat/completions \
  --model YOUR_SERVED_MODEL_ID \
  --prompt 'Explain a cache hit in one short sentence.' \
  --max-tokens 64 \
  --runs 3

These are sequential requests, not a load test. The script sends stream: true and temperature: 0, then prints one JSON object per completed request. It allows 1–20 runs to keep accidental workloads bounded.

Pass the full completion URL, not just http://127.0.0.1:8080 or /v1. If requests cannot connect, start with the connection refused, 503, and 404 troubleshooting guide before interpreting latency.

The probe intentionally ignores environment proxy settings. For a remote server, use an explicitly approved HTTPS endpoint and an appropriate credential. If your network requires a proxy, this version needs adaptation; do not disable TLS verification to make it connect.

3. Read the fields correctly

Field	Meaning	What it does not establish
first_content_ms	Client elapsed time until the first nonempty delta.content for choice 0	Internal engine TTFT or first reasoning-token time
total_ms	Client elapsed time until the completion marker is received and the response closes locally	Pure decode time
content_chunks	Number of nonempty content fragments received	Number of model tokens
output_characters	Python string length summed across those fragments	Token usage or billing
complete	The probe received [DONE]	A correct answer, or an answer that avoided the token cap

first_content_ms is null when a properly completed stream contains no nonempty content. For example, a response might consist only of other fields. Do not convert that result to zero milliseconds or quietly remove it from a report.

The timer uses Python’s time.perf_counter(), which is suited to measuring elapsed intervals. Timing begins immediately before constructing the HTTP opener and opening the request. It therefore includes client overhead, connection work, network delays, server waiting, and processing before answer text arrives.

This is a useful client-observed metric, but it is not a decomposition of those costs. To explain a change, compare it with server-side logs or metrics collected over the same interval.

4. Why the first stream event is not necessarily the first answer

A stream may begin with an assistant-role update or an empty content field. Stopping the stopwatch on that event makes the answer appear to start earlier than it does.

The probe ignores comments, role-only updates, empty strings, and fields other than delta.content. If your model emits reasoning separately, this probe measures the wait for answer content rather than the start of that reasoning. A tool-only response likewise does not count as answer text.

It also waits for complete SSE events. MDN’s SSE documentation explains the blank-line event boundary and how multiple data: lines are combined. Treating each network read as a complete JSON message is not a safe replacement for that parser.

Our test server sends an empty role update, waits 80 milliseconds, then sends content. The automated test checks that the reported first-content delay includes that wait. This validates the measurement logic; the 80-millisecond delay is deliberately simulated, not model performance.

5. Compare settings without mixing different experiments

Before changing anything, record your model file and quantization, llama.cpp build, hardware, server flags, prompt, output limit, client location, and whether other users were active.

Then change one variable and repeat the same request sequence. Keep the raw JSON lines instead of recording only the fastest run:

python3 measure-llama-stream.py \
  --model YOUR_SERVED_MODEL_ID --runs 3 > baseline.jsonl

Use a different output filename for the next configuration so you preserve the baseline. Three requests are a quick diagnostic, not enough to claim a stable tail-latency percentile.

Repeated identical prompts can benefit from cached state. Label the run order and cache conditions; do not automatically call the first request “cold” or every later request “warm.” This probe does not reset the server, clear caches, control slot allocation, or prove a cache hit.

For suspected repeated prefill work, use the KV-cache reuse guide alongside the server logs. For waiting under concurrent traffic, use the parallel slots and queue-metrics guide. The sequential probe here provides a baseline; it does not replace concurrent workload testing.

6. Know when to reject a result

The script raises an error for an HTTP failure, a non-SSE response, malformed event JSON, an error event, or a stream ending without [DONE]. A disconnected stream is not a fast successful answer.

--timeout defaults to 120 seconds. It controls blocking socket operations, not a guaranteed total wall-clock deadline: a server that keeps sending data can run longer. Stop a stuck experiment manually and inspect the server before repeating it.

Finally, do not calculate tokens per second by dividing character or chunk counts by duration. Token throughput needs actual token counts and a clearly defined timing interval. Start with the narrower question this probe can answer honestly: how long did this client wait for answer text, and how long did the completed stream take?