If your coding client cannot connect to llama-server, first separate a connection failure from an HTTP error. Changing model quantization will not repair a wrong port; changing the API key will not start a stopped process.

This guide is for a single-model local server. It is documentation-checked, not a hardware benchmark: we have not run a model inference test for this article. Check your installed version’s help before using flags. Official documentation was checked on September 20, 2026.

Start with one request outside your coding client

Run this on the same machine as the server, outside Docker, replacing the port if yours differs:

curl --noproxy '*' --connect-timeout 3 --max-time 10 \
  -sS -i http://127.0.0.1:8080/health

This is a diagnostic request, not a readiness script. We deliberately show response headers and do not use --fail, so an HTTP error body stays visible. Curl can finish successfully while the server returns an HTTP error. Its process exit code and the response status are different signals. The timeout values here are troubleshooting limits we chose, not recommended model-loading deadlines. See the curl manual.

What you see	Investigate first	Do not assume
Connection refused; no HTTP response	Process, listening address, port	Bad prompt or model ID
HTTP 503 with a loading-model body	Startup progress and server logs	Every 503 means loading
HTTP 404	Request path and the service receiving it	A reachable port is the correct server
Health succeeds but the client fails	Client URL, credentials, request format, network location	Health proves generation works

Save the exact command and output before changing anything. After each change, repeat the same request. That makes the result comparable instead of mixing several possible fixes.

1. Connection refused: find the listener

On macOS, or Linux with lsof installed:

lsof -nP -iTCP:8080 -sTCP:LISTEN

No output means the command did not report a visible listener on that port. Check permissions and the server terminal as well; do not treat this one command as universal proof that no service exists.

Inspect the startup output for an early exit. A model path typo, unsupported option, or allocation failure needs to be fixed before a client can connect. If another process owns the port, identify it rather than killing it blindly. Choose an unused port and update both server and client together.

For a local baseline, substitute a real model path and run your installed binary in the foreground:

llama-server --version
llama-server --help
llama-server -m /absolute/path/to/model.gguf \
  --host 127.0.0.1 --port 8080

The upstream default listener is loopback port 8080. Keep the server local while diagnosing; public exposure is not a connectivity fix. Flags and endpoint behavior are documented in the llama.cpp server README.

2. HTTP 503: read the body before restarting

For the documented single-model health check, a loading-model response uses 503; a loaded, ready server returns 200. The health endpoint does not require an API key. This does not make every 503 a loading signal: a reverse proxy can produce its own error page. Official health endpoint documentation.

Compare the response body with the server log at the same time. If loading is progressing, wait and retry. If the process has exited, repeated client retries cannot repair it. If the log repeats an allocation failure, record the model, context settings, backend, and available memory before changing one setting at a time.

There is no universal safe startup duration. A ten-second HTTP timeout is not evidence that your model should load in ten seconds. Avoid a restart loop that keeps interrupting a slow but advancing startup.

3. HTTP 404: check the full URL, not just the host

Write down the final request URL your client actually sends. Look for a duplicated /v1, a missing proxy prefix, or a request going to a different service on the same machine. Client configuration fields are not interchangeable: a base URL and a full endpoint URL are different inputs.

For an OpenAI-compatible client, llama.cpp documents /v1/chat/completions for chat and /v1/completions for text completions. Its separate /completion route is not the OpenAI-compatible text endpoint. Official API reference.

Once health passes, send one small, non-sensitive request using your client’s documented configuration and the model identifier your deployment serves. Check the response body, not just the status. If that works, restore your real prompt and optional settings one at a time. Do not start the test with tools, a long conversation, and streaming all enabled: that makes the failing layer harder to isolate.

4. Works in the terminal, fails in Docker

Ask where the failing client runs. A container has its own network context; its loopback address is not your host’s loopback. For separate containers, use an appropriate shared network and service address. For host-to-container access, check published ports. See Docker’s networking overview.

Repeat the connectivity check from the client’s actual environment. A successful request on your laptop does not demonstrate that a container, remote editor, or hosted agent can reach the same address. Keep a note with three fields: client location, destination address, and server listener. If those describe different machines or network contexts, fix that mismatch first.

Do not solve the mismatch by disabling authentication or forwarding an unprotected inference endpoint to the Internet. If your server can execute tools, consult the built-in tools safety guide before adding network access.

What counts as fixed?

Use these acceptance checks rather than “the dashboard opened”:

1. The intended server receives a request from the actual client environment.
2. The health response matches the expected state.
3. One minimal generation request returns usable output.
4. Your normal client succeeds with its intended credentials and URL.

If only concurrent requests fail, move on to parallel slots and queue diagnostics. If requests succeed but repeated prompts become slow, use the separate KV-cache reuse guide. Those are different problems from a refused connection.

When asking for help, include the binary version, sanitized launch command, client location, final URL, HTTP status, and relevant log lines. Remove API keys, private prompts, and sensitive filesystem paths first.