If you build an iOS or macOS app on Apple’s Foundation Models framework, AnyLanguageModel 1.0 lets the same session code run against an open-weight model on the device (MLX, llama.cpp GGUF or Core ML), a local Ollama server, or a hosted API such as OpenAI, Anthropic, Gemini or Hugging Face Inference Providers. You switch by changing one import and one model line. Hugging Face shipped 1.0 alongside swift-huggingface 1.0 and swift-xet 1.0, and the release has breaking changes worth checking before you bump an existing app. This guide walks through install, picking a backend, the code for each provider, structured output, tools, token usage, the upgrade, and the errors you are most likely to hit.

Everything below comes from Hugging Face’s AnyLanguageModel 1.0 announcement (October 20, 2026), the 1.0.0 release notes, the project README and the package source on GitHub, checked on October 21, 2026 (Beijing time). We did not build an app on a device for this guide; the Swift snippets are the project’s own examples, lightly combined. Model file sizes come from the Hugging Face Hub file listings.

Key facts (official sources, October 2026):

• What it is: a Swift package that is a drop-in replacement for Apple’s Foundation Models API, with pluggable model providers. Apache-2.0 license.
• Switch: replace import FoundationModels with import AnyLanguageModel.
• Requirements: Swift 6.3+ (Xcode 26.4+), iOS 17+ / macOS 14+ / visionOS 1+, and Linux.
• On-device backends: Apple’s system model, Core ML, MLX and llama.cpp (GGUF). The last three are opt-in package traits (CoreML, MLX, Llama); none is enabled by default.
• Hosted and server backends: OpenAI (Chat Completions and Responses), Open Responses, Anthropic Messages, Google Gemini, Ollama, Apple Private Cloud Compute (OS 27), and any OpenAI-compatible endpoint such as Hugging Face Inference Providers or OpenRouter.
• Breaking in 1.0: JSONValue is now the package’s own type, @Generable enums with associated values use a new encoding (older generated content won’t decode), and swift-huggingface 1.0 is required.

For more local-model walkthroughs, start from the 7minai homepage. If you are building speech features on Apple platforms too, our Apple SpeechAnalyzer guide covers the on-device transcription side.

What changed in 1.0

AnyLanguageModel launched last November as a way to keep Apple’s LanguageModelSession API while choosing your own model. Over the year before 1.0, the announcement says the project added:

• Tool calling with every provider. For llama.cpp it depends on the model’s chat format.
• Guided generation with every provider, including nested @Generable types and arrays of them.
• Image input, using Foundation Models 27’s prompt attachment syntax, including with the system model on iOS, macOS and visionOS 27.
• Any Foundation Models LanguageModel conformer on OS 27, which covers Apple’s Core AI models through a FoundationLanguageModel wrapper.
• More hosted providers: OpenAI Responses, Open Responses, Anthropic with adaptive thinking and effort options, Gemini, Ollama, Private Cloud Compute, and custom base URLs for OpenAI-compatible services.
• Reasoning in the transcript. If you switch providers mid-conversation, reasoning blocks are dropped from requests to a provider that can’t replay them.
• API keys from a closure instead of a string. In the source, apiKey is an @autoclosure returning a String, so you can pass an expression that reads the current token (for example from the Keychain or from a cache your own server refreshes) instead of baking a key into the model at init.
• Linux support, so the same code can run on a server.

The 1.0.0 tag itself (published on GitHub on October 13, 2026) adds JSON Lines streaming, constant and anyOf string guides, Foundation Models 27 prompt attachments, and enums with associated values. Its breaking changes are covered in the upgrade section below.

Choose a backend

The point of the package is that this decision is one line of code, so you can start with whatever is easiest and change later. Here is how the options compare, based on the README:

Backend	Runs where	Trait needed	Image input	Typical use
SystemLanguageModel	On device, OS 26+	none	OS 27+	Apple’s built-in model, zero download
PrivateCloudComputeLanguageModel	Apple servers, OS 27+	none (needs entitlement)	not listed in README	Larger Apple model, same privacy architecture
MLXLanguageModel	On device, Apple silicon	MLX	model-dependent	Hub models in MLX format
LlamaLanguageModel	On device	Llama	model-dependent (needs mmprojPath:)	GGUF files
CoreMLLanguageModel	On device	CoreML	no	Compiled .mlmodelc models
OllamaLanguageModel	Local or remote Ollama server	none	model-dependent	Dev machines, home servers
OpenAILanguageModel, OpenResponsesLanguageModel, AnthropicLanguageModel, GeminiLanguageModel	Hosted	none	yes	Bigger models, server-side tools

For a first local model, the announcement points to small open-weight models that are among the most downloaded on the Hub:

• Gemma 4 E2B from Google in GGUF, using quantization-aware training at Q4_0. The Hub file gemma-4-E2B_q4_0-it.gguf is about 3.35 GB, and the repo also has a 0.99 GB gemma-4-E2B-it-mmproj.gguf projector for image input.
• LFM2.5 2.6B from Liquid AI in GGUF. The Q4_K_M file is about 1.67 GB.
• Qwen3.8 27B for MLX (mlx-community/Qwen3.8-27B-4bit), about 16 GB at 4 bits across three safetensors shards.

Hugging Face says the first two fit on an iPhone, and that Qwen3.8 27B runs on a Mac with 32 GB of memory or more. If you want more context on running Gemma 4 outside an app, see our Gemma 4 12B local guide.

Step 1: Add the package

In a Swift package

Add the dependency to Package.swift. With no traits, you get the system model plus every HTTP provider:

dependencies: [
    .package(url: "https://github.com/huggingface/AnyLanguageModel", from: "1.0.0")
]

To run open-weight models on the device, opt in to the traits you need. Each trait pulls in a heavy dependency, which is why they are off by default:

• CoreML depends on huggingface/swift-transformers.
• MLX depends on ml-explore/mlx-swift-lm.
• Llama requires mattt/llama.swift.

dependencies: [
    .package(
        url: "https://github.com/huggingface/AnyLanguageModel.git",
        from: "1.0.0",
        traits: ["CoreML", "MLX"] // Enable CoreML and MLX support
    )
]

Then add the product to your target as usual:

.product(name: "AnyLanguageModel", package: "AnyLanguageModel")

In an Xcode app project

Xcode doesn’t yet have a built-in way to declare a package dependency with traits. The README’s workaround is a small local package that re-exports the module with the traits you want.

1. From the folder that contains your Xcode project, create the package:

mkdir -p Packages/MyAppKit
cd Packages/MyAppKit
swift package init

2. Replace Packages/MyAppKit/Package.swift with something like this (the README’s example uses an older version number; we set it to 1.0.0 here):

// swift-tools-version: 6.1
import PackageDescription

let package = Package(
    name: "MyAppKit",
    platforms: [
        .macOS(.v14),
        .iOS(.v17),
        .visionOS(.v1),
    ],
    products: [
        .library(
            name: "MyAppKit",
            targets: ["MyAppKit"]
        )
    ],
    dependencies: [
        .package(
            url: "https://github.com/huggingface/AnyLanguageModel",
            from: "1.0.0",
            traits: ["MLX"]
        )
    ],
    targets: [
        .target(
            name: "MyAppKit",
            dependencies: [
                .product(name: "AnyLanguageModel", package: "AnyLanguageModel")
            ]
        )
    ]
)

3. Re-export the module in Sources/MyAppKit/Export.swift:

@_exported import AnyLanguageModel

4. In your project settings, open the Package Dependencies tab, click +, choose Add Local…, and select Packages/MyAppKit.

Your app can now import AnyLanguageModel with MLX enabled. The README points to chat-ui-swift as a working example of this setup.

Step 2: Swap the import and run the system model

If your app already uses Foundation Models, change the import:

- import FoundationModels
+ import AnyLanguageModel

One difference to know: AnyLanguageModel’s session initializers require a model: argument instead of defaulting to SystemLanguageModel.default. The system model needs macOS 26, iOS 26 or visionOS 26 or later:

let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model)

let response = try await session.respond {
    Prompt("Explain quantum computing in one sentence")
}
print(response.content)

Everything after this point keeps the same session code. Only the model line changes.

Step 3: Run an open-weight model on the device

MLX (Apple silicon)

With the MLX trait enabled, pass a Hub model ID:

let model = MLXLanguageModel(modelId: "mlx-community/Qwen3.8-27B-4bit")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("What is the capital of France?")
}

The initializer in the source also accepts an optional hub: client, a directory: and a gpuMemory: setting (default .automatic). For a phone, pick a much smaller MLX model than 27B; the README’s own example uses mlx-community/Qwen3.5-4B-MLX-4bit.

You can tune sampling and the KV cache per request with MLX-specific options:

var options = GenerationOptions(temperature: 0.7)
var mlxOptions = MLXLanguageModel.CustomGenerationOptions.default
mlxOptions.kvCache = .init(
    maxSize: 4096,
    bits: 4,
    groupSize: 64,
    quantizedStart: 128
)
mlxOptions.topP = 0.9
mlxOptions.topK = 40
options[custom: MLXLanguageModel.self] = mlxOptions

let response = try await session.respond(
    to: "Summarize this transcript",
    options: options
)

Sampling fields you leave nil inherit from GenerationOptions, or fall back to MLX’s defaults.

llama.cpp (GGUF)

With the Llama trait enabled, point the model at a GGUF file on disk. One way to get the Gemma 4 E2B file is the Hugging Face CLI (CLI docs):

hf download google/gemma-4-E2B-it-qat-q4_0-gguf gemma-4-E2B_q4_0-it.gguf --local-dir models

For a shipping app you would bundle the file or download it at first launch; either way, the model just needs a path:

let model = LlamaLanguageModel(modelPath: "/path/to/model.gguf")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Translate 'hello world' to Spanish")
}

The initializer is init(modelPath:gpuLayers:mmprojPath:). gpuLayers defaults to LlamaLanguageModel.defaultGPULayerCount, and mmprojPath is the multimodal projector GGUF that matches the model. Without it the model is text-only, so for Gemma 4 E2B image input you would also download gemma-4-E2B-it-mmproj.gguf and pass its path.

Runtime settings go in per-request custom options. These are from the README:

var options = GenerationOptions(temperature: 0.8)
options[custom: LlamaLanguageModel.self] = .init(
    contextSize: 4096,        // Context window size
    batchSize: 512,           // Batch size for evaluation
    threads: 8,               // Number of threads
    seed: 42,                 // Random seed for deterministic output
    temperature: 0.7,         // Sampling temperature
    topK: 40,                 // Top-K sampling
    topP: 0.95,               // Top-P (nucleus) sampling
    repeatPenalty: 1.2,       // Penalty for repeated tokens
    repeatLastN: 128          // Number of tokens to consider for repeat penalty
)

let response = try await session.respond(
    to: "Write a story",
    options: options
)

The README also shows assistantPrefill: "<think></think>", which sets the text the response continues from (its example pre-fills an empty think block), plus frequencyPenalty, presencePenalty and mirostat options. Every field is optional.

Core ML

With the CoreML trait, load a compiled model:

let model = CoreMLLanguageModel(url: URL(fileURLWithPath: "path/to/model.mlmodelc"))

Core ML has no image input in this package, and it doesn’t send transcript history (prompt-only behavior), so it fits single-shot tasks better than chat.

Ollama

Ollama needs no trait. It’s handy while you develop on a Mac, or for an app that talks to a home server:

// Default: connects to http://localhost:11434
let model = OllamaLanguageModel(model: "qwen3.5") // `ollama pull qwen3.5:9b`

// Custom endpoint
let remote = OllamaLanguageModel(
    endpoint: URL(string: "http://remote-server:11434")!,
    model: "gemma4"
)

Model-specific parameters go in a dictionary:

var options = GenerationOptions(temperature: 0.8)
options[custom: OllamaLanguageModel.self] = [
    "seed": .int(42),
    "num_ctx": .int(4096)
]

Step 4: Use a hosted model with the same code

When a model is too big for the device, keep the session code and swap the model. For open-weight models through Hugging Face Inference Providers, the announcement uses the OpenAI-compatible router:

let model = OpenAILanguageModel(
    baseURL: URL(string: "https://router.huggingface.co/v1/")!,
    apiKey: ProcessInfo.processInfo.environment["HF_TOKEN"]!,
    model: "openai/gpt-oss-120b",
    apiVariant: .chatCompletions
)

The other providers follow the same pattern. From the README:

let claude = AnthropicLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]!,
    model: "claude-sonnet-5"
)

let gemini = GeminiLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["GEMINI_API_KEY"]!,
    model: "gemini-3.8-flash"
)

let router = OpenResponsesLanguageModel(
    baseURL: URL(string: "https://openrouter.ai/api/v1/")!,
    apiKey: ProcessInfo.processInfo.environment["OPEN_RESPONSES_API_KEY"]!,
    model: "openai/gpt-5.6"
)

Provider-specific features stay available through custom options. For example, Anthropic adaptive thinking with an effort level (.low, .medium, .high, .extraHigh, .max; support varies by model):

var options = GenerationOptions(maximumResponseTokens: 8192)
options[custom: AnthropicLanguageModel.self] = .init(
    thinking: .adaptive(display: .omitted),
    effort: .medium
)

Gemini’s server-side tools (.googleSearch, .googleMaps, .codeExecution, .urlContext) are set the same way, but the README notes they are not available as client Tools for other models.

Don’t ship API keys in the app. Environment variables are fine for development and tests. For production the README describes two options: let users bring their own key and store it in the Keychain (it warns that App Review has often rejected this model), or route requests through your own server and hand the app short-lived, scoped tokens. Because apiKey takes a closure-style argument, the second option fits naturally: pass an expression that returns the current token.

Step 5: Get typed output with guided generation

All on-device backends and the hosted providers support guided generation. You describe a Swift type and get an instance back instead of parsing text:

@Generable(description: "Basic profile information about a cat")
struct CatProfile {
    // A guide isn't necessary for basic fields.
    var name: String

    @Guide(description: "The age of the cat", .range(0...20))
    var age: Int

    @Guide(description: "A one sentence profile about the cat's personality")
    var profile: String
}

let session = LanguageModelSession(model: model)
let response = try await session.respond(
    to: "Generate a cute rescue cat",
    generating: CatProfile.self
)
print(response.content)

This is the part worth testing across backends before you commit to a small local model. The API is the same everywhere, but a 2B model will follow a nested schema less reliably than a hosted one.

Step 6: Add tools and control when they run

Tools use the Foundation Models Tool protocol:

struct WeatherTool: Tool {
    let name = "getWeather"
    let description = "Retrieve the latest weather information for a city"

    @Generable
    struct Arguments {
        @Guide(description: "The city to fetch the weather for")
        var city: String
    }

    func call(arguments: Arguments) async throws -> String {
        "The weather in \(arguments.city) is sunny and 72°F / 23°C"
    }
}

let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
    Prompt("How's the weather in Cupertino?")
}
print(response.content)

Two additions matter for real apps.

Approve tool calls before they run. Assign a ToolExecutionDelegate (an AnyLanguageModel extension). Its toolCallDecision(for:in:) method can return .execute, .stop to halt after the tool calls, or .provideOutput(...) to skip running the tool. The README suggests this as the place to show a confirmation dialog.

Force or forbid tool use. Set toolCallingMode on GenerationOptions:

let options = GenerationOptions(
    samplingMode: .greedy,
    toolCallingMode: .required
)

Hosted providers map this to their own tool-choice field (for example "tool_choice": "required" for OpenAI and {"type": "any"} for Anthropic). Ollama, MLX, llama.cpp and Core ML have no such setting: they honor .disallowed by leaving tools out of the prompt, but treat .required like .allowed. With .required, the model keeps calling tools until a tool throws, the delegate stops it, the tools are removed, or the round count passes maximumToolRounds and the response ends with LanguageModelSession.ToolRoundLimitExceeded.

Step 7: Stream responses and track tokens

Streaming uses streamResponse(to:), where each snapshot carries the content so far:

for try await snapshot in session.streamResponse(to: "Explain your approach") {
    print(snapshot.content)
}

Token usage is on every response and accumulates on the session:

let response = try await session.respond(to: "Explain how rainbows form.")
let usage = response.usage

print("Input tokens:", usage.input.totalTokenCount)
print("Cached input tokens:", usage.input.cachedTokenCount)
print("Output tokens:", usage.output.totalTokenCount)
print("Session total:", session.usage.totalTokenCount)

OpenAI, Open Responses, Anthropic, Gemini, Ollama, MLX, llama.cpp and Core ML report usage. Counts a provider doesn’t report default to zero. MLX and llama.cpp report the reused prompt prefix as cached input, while Core ML always reports zero cached tokens because it resets model state. A session restored from a saved transcript starts with zero accumulated usage.

Upgrading from 0.x to 1.0

The 1.0.0 release notes list three breaking changes:

1. JSONValue is now AnyLanguageModel’s own type instead of a typealias for JSONSchema’s JSONValue. It has the same cases, accessors and literal conformances, but not description, string interpolation, or isCompatible(with:strict:). If you pass values between the two types, convert them explicitly.
2. @Generable enums with associated values are generated the way Foundation Models does it: one object per case, with a type naming the case and a typed property per value. Content generated by an earlier version won’t decode. If you persisted structured output from 0.x, plan a migration or regenerate it.
3. swift-huggingface 1.0 is required. That release also has breaking changes: it requires swift-xet 1.0, HTTPClientError is now a struct you switch on via error.code, and Provider, Capability and Bucket.Region are now structs, so exhaustive switch statements need a default case. See the swift-huggingface 1.0.0 notes if your code uses it directly.

Then update the requirement to from: "1.0.0" and rebuild.

Troubleshooting

“exhausted attempts to resolve the dependencies graph” when traits are enabled. This is a Swift Package Manager bug. The README’s workaround is to add each trait’s underlying dependency directly, only for the traits you enable:

dependencies: [
    .package(
        url: "https://github.com/huggingface/AnyLanguageModel.git",
        from: "1.0.0",
        traits: ["CoreML", "MLX", "Llama"]
    ),
    .package(url: "https://github.com/huggingface/swift-transformers", from: "1.0.0"), // CoreML
    .package(url: "https://github.com/ml-explore/mlx-swift-lm", from: "2.25.5"),       // MLX
    .package(url: "https://github.com/mattt/llama.swift", from: "2.0.0"),              // Llama
]

MLXLanguageModel or LlamaLanguageModel is not found. The trait isn’t enabled. In an Xcode project, check that the app depends on the local shim package and not on AnyLanguageModel directly.

The system model throws when the prompt has an image. SystemLanguageModel bridges prompt attachments only in apps built with Swift 6.4 or later running on macOS, iOS or visionOS 27 or later, and the bridge needs local file URLs. On earlier runtimes, and on tvOS and watchOS, it throws.

SystemLanguageModel.Error.dynamicInstructionsUnavailable. Dynamic instructions follow the Foundation Models 27 API, so the system model supports them only when built with Swift 6.4+ and running on OS 27+, and not on tvOS.

An Anthropic request with toolCallingMode: .required is rejected. Anthropic rejects forced tool use ({"type": "any"}) with manual extended thinking (thinking: .enabled(budgetTokens:)) and on models that don’t support forced tool use.

Image prompts do nothing on a local model. Use a vision-capable model. For llama.cpp, pass the matching projector with mmprojPath:.

MLX tests fail under swift test. The README says MLX tests must run with xcodebuild because of Metal library loading, after enabling the MLX trait by default in Package.swift, with environment variables passed using a TEST_RUNNER_ prefix (for example TEST_RUNNER_HF_TOKEN).

FAQ

Is AnyLanguageModel an Apple project? No. It’s an open-source Hugging Face project (Apache-2.0) that mirrors Apple’s Foundation Models API. Anything it adds beyond Apple’s API is marked as an extension in its documentation, and code that uses an extension won’t compile against Foundation Models on OS 26.

Do I need iOS 26 to use it? No. The package supports iOS 17+, macOS 14+ and visionOS 1+. Only the Apple-provided models need newer OS versions: the system model needs OS 26, and Private Cloud Compute needs OS 27 plus its entitlement.

Does it work on Linux? Yes, according to the README and the announcement, so you can share code between an app and a Swift server. The Apple system models and MLX are Apple-platform features, so on a server you would mainly use the HTTP providers; check the README before relying on another local backend there.

Which local model should I start with on an iPhone? Hugging Face suggests Gemma 4 E2B (about 3.35 GB at Q4_0) or LFM2.5 2.6B (about 1.67 GB at Q4_K_M), both as GGUF through the Llama trait. Test guided generation and tool calls with your own prompts before you choose, since small models vary most on those.

Can I mix providers in one conversation? Yes. The transcript is shared, and reasoning entries a new provider can’t replay are left out of its requests while staying in the transcript for display and persistence.

What’s next for the project? The announcement says the same approach is coming to other model types, teasing “models that decide, and models that listen.”

Sources

• Hugging Face blog: AnyLanguageModel 1.0: One Year of Local Models on Apple Platforms
• GitHub: AnyLanguageModel 1.0.0 release notes and README
• GitHub: swift-huggingface 1.0.0 release notes
• Hugging Face Hub: google/gemma-4-E2B-it-qat-q4_0-gguf, LiquidAI/LFM2.5-2.6B-GGUF, mlx-community/Qwen3.8-27B-4bit
• Apple: Foundation Models documentation