Tools · · 2 min read

llama-server --tools all: Built-in Agent Tools for Any Local GGUF Model

llama.cpp's Web UI can give a local model built-in file, search, edit, and shell tools. Current builds also support Docker, Podman, and SSH tool runtimes, making isolation possible instead of running everything on the host.

A Reddit thread hit r/LocalLLaMA on May 23, 2026 with a finding that surprised even the local-LLM regulars: llama-server’s Web UI now has built-in native tool execution. Current builds expose read_file, file_glob_search, grep_search, exec_shell_command, write_file, edit_file, and get_info.

The flag is one line:

llama-server -m model.gguf --tools all

The server’s README documents the activation flag (--tools name1,name2,... to enable individual tools, or all for everything). Tools are exposed through an internal /tools REST endpoint used by the Web UI; the project warns that the endpoint may change and says not to enable tools in untrusted environments.

August 2026 update: isolation is now built in

The original release had a serious limitation: without another boundary, file and shell tools ran with the same host privileges as llama-server. That is no longer the only option. Current builds add:

  • --tools-runtime docker:<image> or podman:<image> to start an isolated tool container.
  • --tools-runtime docker-container:<id> or podman-container:<id> to attach to a container whose mounts, network, limits, and user you control.
  • --tools-runtime ssh:<target> to move execution to a dedicated POSIX machine. SSH is remote execution, not automatic sandboxing.

What this means if you’re a builder

1. The built-in Web UI can now be a local coding agent. A compatible model can search files, edit code, run tests, and inspect the results without a separate file-tools server. This does not mean every OpenAI-compatible client automatically gains an agent loop: --tools is specifically documented as llama.cpp’s Web UI tool layer, while external clients still need to integrate tool execution themselves.

2. Don’t expose port 8080 directly. Keep the server on its default 127.0.0.1 binding. Enabling tools limits CORS to localhost by default, but CORS and API keys do not protect you from a malicious prompt that an authenticated user feeds to the agent.

3. Least privilege still beats all. Start with read_file,file_glob_search,grep_search,get_info. Add write and shell tools only for a bounded task, preferably inside a container with no network and only one mounted project directory.

4. Test the loop, not just the model. The practical bar is: can it read repository instructions, run the smallest relevant test, edit one file, re-run the test, summarize the diff, and stop? Tool-call formatting and stop behavior matter as much as benchmark scores.

The bigger picture

llama.cpp is moving beyond inference into a local agent runtime: the Web UI can use built-in tools, current builds can load stdio MCP servers, and tool execution can be moved into a container or dedicated machine. The feature remains experimental, but the security model is materially better than it was in May.

For the exact commands and a container setup that exposes only one disposable workspace, follow the safe llama-server built-in tools guide. For a model to serve behind it, see Qwen3.6 local coding.

Sources

Source: llama.cpp README