Read time: ~8 minutes. What you’ll build: a local llama-server agent that can inspect a codebase, search files, edit code, and run commands — first with a low-risk read-only tool set, then inside a locked-down Docker workspace instead of directly on your host.

Version note: built-in tools and --tools-runtime are experimental and moving quickly. The commands below were checked against the official llama.cpp server documentation on August 22, 2026. Confirm your binary with llama-server --help before copying flags into production.

TL;DR — the safe starting point

Do not begin with --tools all on your normal development account. Start with the read-only tools and keep the server on localhost:

llama-server \
  -m /path/to/model.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  --tools read_file,file_glob_search,grep_search,get_info

Open http://127.0.0.1:8080, point the agent at a disposable project, and ask it to explain the repository before giving it write or shell access.

When you need the full coding loop, run the tools in an existing container whose only writable mount is the project:

docker run -d --name llama-agent-workspace \
  --network none \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=256m \
  --user "$(id -u):$(id -g)" \
  -v "$PWD:/workspace:rw" \
  -w /workspace \
  ubuntu:24.04 sleep infinity

CONTAINER_ID="$(docker inspect --format '{{.Id}}' llama-agent-workspace)"

llama-server \
  -m /path/to/model.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  --tools all \
  --tools-runtime "docker-container:${CONTAINER_ID}"

Replace ubuntu:24.04 with a project image that contains the compiler, test runner, Node/Python version, and other tools your repository needs. The important boundary is the mount: the agent sees /workspace, not your home directory, SSH keys, browser profile, or cloud credentials.

If you are catching up on the feature itself, the original llama-server built-in tools announcement now includes an August update covering the newer isolation backends.

---

1. What --tools actually enables

llama-server now exposes built-in file and command tools to its Web UI. The current official list is:

Tool	Capability	Risk
read_file	Read a file or line range	Can expose secrets
file_glob_search	Find files by glob pattern	Reveals project structure
grep_search	Search file contents	Can discover credentials
exec_shell_command	Run a shell command	Arbitrary code execution
write_file	Create or replace files	Destructive writes
edit_file	Modify existing files	Destructive writes
get_info	Inspect the tool environment	Low risk

Enable a comma-separated allowlist:

llama-server -m model.gguf \
  --tools read_file,file_glob_search,grep_search,get_info

Or enable everything:

llama-server -m model.gguf --tools all

The second command is convenient, but it gives the agent a shell and write access in the tool runtime. Without --tools-runtime, that runtime is the same host environment and privilege level as llama-server.

This feature is primarily wired into llama.cpp’s Web UI. GET /tools returns the available tool definitions and POST /tools invokes a tool, but the project explicitly describes that REST surface as internal and subject to change. Do not build a long-lived integration around /tools without pinning a llama.cpp build.

2. Check whether your build supports isolation

Run:

llama-server --help | grep -E -- '--tools|--tools-runtime|--agent'

You want to see both --tools and --tools-runtime. Current builds document these runtime forms:

docker:<image>
podman:<image>
docker-container:<container-id>
podman-container:<container-id>
ssh:<user@host-or-ssh-alias>

The image forms start a container and clean it up when the server exits. The *-container: forms attach to a container you created, so you control its mounts, network, resource limits, user, and lifecycle. For a coding agent, the attach form is easier to audit and is the one used in this guide.

ssh: moves execution to another POSIX machine; it does not create a sandbox by itself. The remote account can still do anything its permissions allow. Use a dedicated VM or restricted account, key authentication, and a previously trusted host key.

If your binary has --tools but not --tools-runtime, update llama.cpp before enabling shell or write tools. Isolation landed after the first built-in tools release, so older packages may expose the dangerous half without the safer execution backend.

3. Run a read-only repository review first

Launch the server from the project you want to inspect:

cd /path/to/disposable-project

llama-server \
  -m /path/to/model.gguf \
  --host 127.0.0.1 \
  --tools read_file,file_glob_search,grep_search,get_info

Then open the Web UI and try a bounded task:

Inspect this repository without changing files. Find the test command,
summarize the architecture, and list the three highest-risk modules.
Show the files that support each conclusion.

The point of this first pass is not only security. It tells you whether the model’s tool-calling template works. A model can be strong at code completion and still emit malformed tool calls, loop over the same search, or ignore tool results.

Useful checks:

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/tools

If /tools is empty, confirm the flag reached the process. If the model describes a tool call as prose instead of executing it, verify that the GGUF includes a compatible chat template and that your build has Jinja tool-call support enabled.

4. Give the agent a container, not your laptop

For edits and test execution, create a dedicated container before launching the model server:

docker run -d --name llama-agent-workspace \
  --network none \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=256m \
  --memory 4g \
  --cpus 4 \
  --pids-limit 256 \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --user "$(id -u):$(id -g)" \
  -v "$PWD:/workspace:rw" \
  -w /workspace \
  ubuntu:24.04 sleep infinity

Then attach llama-server’s tools to it:

CONTAINER_ID="$(docker inspect --format '{{.Id}}' llama-agent-workspace)"

llama-server \
  -m /path/to/model.gguf \
  --host 127.0.0.1 \
  --api-key "replace-with-a-long-random-key" \
  --tools all \
  --tools-runtime "docker-container:${CONTAINER_ID}"

What the restrictions do:

• --network none blocks downloads and simple data exfiltration from the tool container.
• --read-only makes the container filesystem immutable except for explicit writable mounts and /tmp.
• -v "$PWD:/workspace:rw" exposes one project, not your whole machine.
• --cap-drop ALL and no-new-privileges remove common escalation paths.
• CPU, memory, and process limits reduce runaway command damage.
• Running as your numeric user avoids root-owned edits in the mounted repository.

This is containment, not magic. A malicious command can still delete or corrupt files under /workspace. Work on a clean Git branch, keep uncommitted secrets outside the mount, and review the diff before merging.

5. Why localhost and API keys are not enough

When --tools or --agent is enabled, llama.cpp changes the default CORS policy to localhost. That is a useful guardrail, but it is not a complete security boundary.

An API key answers “who can reach the server?” It does not answer “should this prompt be allowed to run rm, read .env, or upload source code?” Prompt injection can arrive through a pasted issue, a repository file, a generated link, or content the agent reads while solving a task.

Use all four layers:

1. Network boundary: keep --host 127.0.0.1; do not expose port 8080 directly.
2. Tool allowlist: enable only the tools needed for the current task.
3. Execution boundary: use --tools-runtime with a purpose-built container or dedicated remote machine.
4. Human boundary: keep confirmation prompts enabled and inspect commands and diffs.

Avoid clicking “always allow all tools” in the Web UI on a long-running instance. A July 2026 security report showed that an auto-submitted Web UI query could trigger a previously auto-approved shell tool. The reported mitigations were to disable tools or the Web UI, retain confirmation, and sandbox execution — not to rely on the API key alone.

6. A practical coding-agent test

Use a throwaway branch and give the model a task with a clear stop condition:

git switch -c test/llama-agent

Prompt:

Read the repository instructions first. Run the smallest relevant test suite.
Fix only the first failing test. Re-run that test, summarize the diff, and stop.
Do not install packages or access the network.

Review after it stops:

git status --short
git diff --check
git diff

This test reveals more than asking it to “build an app.” You learn whether the model can follow repository instructions, select the right files, interpret command output, make a minimal edit, verify it, and stop instead of expanding scope.

If first-token latency becomes the bottleneck during repeated agent turns, pair this setup with the llama-server KV cache reuse guide. For a model that fits on a single 24 GB GPU or 32 GB Mac, the Qwen3.6 local coding setup is a practical starting point.

7. Common mistakes

“I enabled --tools all, but nothing happens”

Use the built-in Web UI first and check GET /tools. The /v1/chat/completions endpoint supports function calling, but --tools is specifically documented as the Web UI’s local tool layer; an external OpenAI-compatible client still needs its own agent loop unless it integrates llama.cpp’s tool surface.

“The container cannot see my code”

The auto-created docker:<image> runtime does not automatically grant your host repository. Use an existing container with an explicit project mount, then attach with docker-container:<id>.

“Tests fail because the command is missing”

ubuntu:24.04 is only a safe minimal example. Build a project-specific image with the exact Node, Python, compiler, package manager, and test dependencies. Keep credentials and Docker socket mounts out of that image.

“I bound the server to 0.0.0.0 for convenience”

Undo that unless you have a deliberate authenticated reverse-proxy design. With write and shell tools enabled, an exposed server is much closer to remote command execution infrastructure than a normal inference endpoint.

The takeaway

llama-server --tools all can turn a local GGUF into a useful coding agent, but the safe default is not “all tools on the host.” Start read-only, verify the model can use tools, and move write/shell execution behind --tools-runtime in a container that sees only a disposable project workspace. Keep the server on localhost, preserve confirmations, and review every diff.

That gives you the useful part of a local coding agent — repository search, edits, commands, and tests — without handing an experimental Web UI unrestricted access to your daily-driver machine.

Sources

• llama.cpp server README — current --tools, --tools-runtime, --agent, CORS, Web UI, /tools, and function-calling documentation
• llama.cpp server tools implementation — current tool definitions, permissions, output limits, and runtime implementation
• PR #26507 — initial Docker isolation backend for built-in tools
• PR #26774 — Podman and SSH runtime support plus runtime hardening, merged August 10, 2026
• Issue #25790 — Web UI query/prompt-injection report and mitigations; open and labeled unconfirmed when checked
• Commands and available tools checked against llama.cpp master documentation on August 22, 2026. Experimental flags can change; verify with your installed build’s --help output.