foundation-models

Original source / Raw SKILL.md

---
name: foundation-models
description: Use when implementing on-device AI with Apple's Foundation Models framework (iOS 26+), building summarization/extraction/classification features, or using @Generable for type-safe structured output.
---

# Foundation Models

Apple's on-device AI framework providing access to a 3B parameter language model for summarization, extraction, classification, and content generation. Runs entirely on-device with no network required.

## Overview

Foundation Models enable intelligent text processing directly on device without server round-trips, user data sharing, or network dependencies. The core principle: leverage on-device AI for specific, contained tasks (not for general knowledge).

## Reference Loading Guide

**ALWAYS load reference files if there is even a small chance the content may be required.** It's better to have the context than to miss a pattern or make a mistake.

| Reference | Load When |
|-----------|-----------|
| **[Getting Started](references/getting-started.md)** | Setting up LanguageModelSession, checking availability, basic prompts |
| **[Structured Output](references/structured-output.md)** | Using `@Generable` for type-safe responses, `@Guide` constraints |
| **[Tool Calling](references/tool-calling.md)** | Integrating external data (weather, contacts, MapKit) via Tool protocol |
| **[Streaming](references/streaming.md)** | AsyncSequence for progressive UI updates, PartiallyGenerated types |
| **[Troubleshooting](references/troubleshooting.md)** | Context overflow, guardrails, errors, anti-patterns |

## Core Workflow

1. Check availability with `SystemLanguageModel.default.availability`
2. Create `LanguageModelSession` with optional instructions
3. Choose output type: plain String or @Generable struct
4. Use streaming for long generations (>1 second)
5. Handle errors: context overflow, guardrails, unsupported language

## Model Capabilities

| Use Case | Foundation Models? | Alternative |
|----------|-------------------|-------------|
| Summarization | Yes | - |
| Extraction (key info) | Yes | - |
| Classification | Yes | - |
| Content tagging | Yes (built-in adapter) | - |
| World knowledge | No | ChatGPT, Claude, Gemini |
| Complex reasoning | No | Server LLMs |

## Platform Requirements

- iOS 26+, macOS 26+, iPadOS 26+, visionOS 26+
- Apple Intelligence-enabled device (iPhone 15 Pro+, M1+ iPad/Mac)
- User opted into Apple Intelligence

## Common Mistakes

1. **Using Foundation Models for world knowledge** — The 3B model is trained for on-device tasks only. It won't know current events, specific facts, or "who is X". Use ChatGPT/Claude for that. Keep prompts to: summarizing user's own content, extracting info, classifying text.

2. **Blocking the main thread** — LanguageModelSession calls must run on a background thread or async context. Blocking the main thread locks UI. Always use `Task { }` or background queue.

3. **Ignoring context overflow** — The model has finite context. If the user pastes a 50KB document, it will fail silently or truncate. Check input length and trim/truncate proactively.

4. **Forgetting to check availability** — Not all devices support Foundation Models. Check `SystemLanguageModel.default.availability` before using. Graceful degradation is required.

5. **Ignoring guardrails** — The model won't answer harmful queries. Instead of fighting it, design prompts that respect safety guidelines. Rephrasing requests usually works.


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### references/getting-started.md

```markdown
# Getting Started with Foundation Models

## Availability Check

Always check before creating a session:

```swift
import FoundationModels

switch SystemLanguageModel.default.availability {
case .available:
    print("Foundation Models available")
case .unavailable(let reason):
    print("Unavailable: \(reason)")
}
```

**Requirements:** iPhone 15 Pro+, M1+ iPad/Mac, supported region, user opted in.

## Creating a Session

```swift
// Basic
let session = LanguageModelSession()

// With instructions (define model's role)
let session = LanguageModelSession(instructions: """
    You are a helpful travel assistant.
    Respond concisely.
    """
)
```

**Important:** Never interpolate user input into instructions (security risk).

## Basic Text Generation

```swift
let session = LanguageModelSession()
let response = try await session.respond(to: "Summarize this article...")
print(response.content)
```

## Multi-Turn Conversations

Sessions retain transcript automatically:

```swift
let session = LanguageModelSession()

let first = try await session.respond(to: "Write a haiku about fishing")
// "Silent waters gleam..."

let second = try await session.respond(to: "Now one about golf")
// Model remembers context from first turn
```

## Preventing Concurrent Requests

```swift
Button("Generate") {
    Task {
        result = try await session.respond(to: "Write a haiku").content
    }
}
.disabled(session.isResponding)
```

## Prewarming for Performance

First generation takes 1-2s to load. Prewarm before user interaction:

```swift
class ViewModel: ObservableObject {
    private var session: LanguageModelSession?

    init() {
        Task { self.session = LanguageModelSession() }
    }

    func generate(prompt: String) async throws -> String {
        try await session!.respond(to: prompt).content
    }
}
```

## SwiftUI Availability Pattern

```swift
struct AIFeatureView: View {
    var body: some View {
        switch SystemLanguageModel.default.availability {
        case .available:
            AIContentView()
        case .unavailable:
            Text("AI features require Apple Intelligence")
        }
    }
}
```

## Model Specifications

- **Context Window:** 4096 tokens (~12,000 chars)
- **Good for:** Summarization, extraction, classification
- **Not for:** World knowledge, complex reasoning

```

### references/structured-output.md

```markdown
# Structured Output with @Generable

## Why @Generable Over JSON

Manual JSON parsing fails unpredictably:

```swift
// BAD - Model might output wrong keys or invalid JSON
let response = try await session.respond(to: "Generate person as JSON")
let person = try JSONDecoder().decode(Person.self, from: data) // CRASHES!
```

@Generable uses constrained decoding - model can only generate valid structure:

```swift
@Generable
struct Person {
    let name: String
    let age: Int
}

let response = try await session.respond(
    to: "Generate a person",
    generating: Person.self
)
let person = response.content // Guaranteed valid Person
```

## Supported Types

```swift
@Generable
struct Example {
    let text: String           // Primitives
    let count: Int
    let isActive: Bool
    let items: [String]        // Arrays
    let plan: DayPlan          // Nested @Generable types
}

@Generable
enum Encounter {               // Enums with associated values
    case order(item: String)
    case complaint(reason: String)
}
```

## @Guide Constraints

```swift
@Generable
struct Character {
    @Guide(description: "A full name")
    let name: String

    @Guide(.range(1...10))
    let level: Int

    @Guide(.count(4))
    var searchTerms: [String]   // Exactly 4 items

    @Guide(.maximumCount(3))
    let topics: [String]        // Up to 3 items
}
```

**Regex patterns:**
```swift
@Guide(Regex {
    ChoiceOf { "Mr"; "Mrs" }
    ". "
    OneOrMore(.word)
})
let name: String  // Output: "Mrs. Brewster"
```

## Property Order Matters

Properties generate in declaration order. Put summaries last:

```swift
@Generable
struct Itinerary {
    var destination: String  // Generated first
    var days: [DayPlan]      // Generated second
    var summary: String      // Generated last (references days)
}
```

## Skip Schema on Subsequent Requests

```swift
// First request - schema inserted automatically
let first = try await session.respond(to: "Generate person", generating: Person.self)

// Subsequent - skip schema for 10-20% speedup
let second = try await session.respond(
    to: "Generate another",
    generating: Person.self,
    options: GenerationOptions(includeSchemaInPrompt: false)
)
```

## Content Tagging Adapter

```swift
let session = LanguageModelSession(
    model: SystemLanguageModel(useCase: .contentTagging)
)

@Generable
struct TagResult {
    @Guide(.maximumCount(5))
    let topics: [String]
}

let result = try await session.respond(to: article, generating: TagResult.self)
```

```

### references/tool-calling.md

```markdown
# Tool Calling

## Why Tools Are Needed

The 3B model will hallucinate external data:

```swift
// BAD - Model will make up weather
let response = try await session.respond(to: "Weather in Tokyo?")
```

Tools let the model call your code to fetch real data.

## Tool Protocol

```swift
protocol Tool {
    var name: String { get }
    var description: String { get }
    associatedtype Arguments: Generable
    func call(arguments: Arguments) async throws -> ToolOutput
}
```

## Weather Tool Example

```swift
struct GetWeatherTool: Tool {
    let name = "getWeather"
    let description = "Get current weather for a city"

    @Generable
    struct Arguments {
        @Guide(description: "City name")
        var city: String
    }

    func call(arguments: Arguments) async throws -> ToolOutput {
        let places = try await CLGeocoder().geocodeAddressString(arguments.city)
        let weather = try await WeatherService.shared.weather(for: places.first!.location!)
        return ToolOutput("Temperature: \(weather.currentWeather.temperature.value)F")
    }
}
```

## Using Tools

```swift
let session = LanguageModelSession(
    tools: [GetWeatherTool()],
    instructions: "Help with weather forecasts."
)

let response = try await session.respond(to: "What's the temperature in Cupertino?")
// Model calls GetWeatherTool, uses real data in response
```

## How It Works

1. User prompt arrives
2. Model decides it needs external data
3. Model generates tool call with @Generable arguments
4. Framework calls your `call()` method
5. Tool output inserted into transcript
6. Model generates final response using real data

## Stateful Tools

Use `class` to track state across calls:

```swift
class FindContactTool: Tool {
    var pickedContacts = Set<String>()

    func call(arguments: Arguments) async throws -> ToolOutput {
        contacts.removeAll(where: { pickedContacts.contains($0.name) })
        guard let picked = contacts.randomElement() else {
            return ToolOutput("No more contacts")
        }
        pickedContacts.insert(picked.name)
        return ToolOutput(picked.name)
    }
}
```

## Multiple Tools

```swift
let session = LanguageModelSession(
    tools: [GetWeatherTool(), FindRestaurantTool(), FindHotelTool()],
    instructions: "Plan travel itineraries."
)
// Model autonomously decides which tools to call
```

## ToolOutput Options

```swift
// Natural language
return ToolOutput("Temperature is 71F")

// Structured
return ToolOutput(GeneratedContent(properties: ["temperature": 71]))
```

## Tool Naming

- Short, readable: `getWeather`, `findContact`
- Use verbs: `get`, `find`, `fetch`
- Concise descriptions (they're in the prompt)

## When to Use Tools

**Use for:** Weather, MapKit, Contacts, Calendar, external APIs
**Don't use for:** Data already in prompt, simple calculations

```

### references/streaming.md

```markdown
# Streaming Responses

## Why Stream

Without streaming, users wait 3-5 seconds seeing nothing. With streaming, first content appears in 0.2s.

## PartiallyGenerated Type

@Generable auto-generates a streaming type with optional properties:

```swift
@Generable
struct Itinerary {
    var name: String
    var days: [DayPlan]
}

// Generated automatically:
// Itinerary.PartiallyGenerated {
//     var name: String?
//     var days: [DayPlan]?
// }
```

## Basic Streaming

```swift
let stream = session.streamResponse(
    to: "Generate 3-day itinerary",
    generating: Itinerary.self
)

for try await partial in stream {
    print(partial.name)  // Fills in progressively
    print(partial.days)
}
```

## SwiftUI Integration

```swift
struct ItineraryView: View {
    @State private var itinerary: Itinerary.PartiallyGenerated?

    var body: some View {
        VStack {
            if let name = itinerary?.name {
                Text(name).font(.title)
            }

            if let days = itinerary?.days {
                ForEach(days, id: \.self) { day in
                    DayView(day: day)
                }
            }

            Button("Generate") {
                Task {
                    let stream = session.streamResponse(
                        to: "Generate itinerary",
                        generating: Itinerary.self
                    )
                    for try await partial in stream {
                        self.itinerary = partial
                    }
                }
            }
        }
    }
}
```

## Animations

```swift
if let name = itinerary?.name {
    Text(name)
        .transition(.opacity)
}

if let days = itinerary?.days {
    ForEach(days, id: \.id) { day in  // Use stable ID, not indices
        DayView(day: day)
            .transition(.slide)
    }
}
.animation(.default, value: itinerary)
```

## Property Order for UX

Properties generate in declaration order. Put quick/important ones first:

```swift
@Generable
struct Article {
    var title: String      // Shows in 0.2s
    var summary: String    // Shows in 0.8s
    var fullText: String   // Shows in 2.5s (user already engaged)
}
```

Summaries should be last - they reference earlier content.

## When to Stream

**Stream:** Itineraries, stories, long descriptions, any >1 second generation
**Skip:** Quick classification, content tagging, short responses

```

### references/troubleshooting.md

```markdown
# Troubleshooting Foundation Models

## Anti-Patterns

### Using for World Knowledge
Model is 3B parameters - optimized for summarization/extraction, NOT encyclopedic knowledge.
```swift
// BAD - Will hallucinate
session.respond(to: "What's the capital of France?")
```
**Fix:** Use server LLMs or provide data via Tools.

### Manual JSON Parsing
```swift
// BAD - Crashes on wrong keys/invalid JSON
let json = try await session.respond(to: "Generate person as JSON")
JSONDecoder().decode(Person.self, from: json.data)
```
**Fix:** Use @Generable for guaranteed structure.

### Blocking Main Thread
```swift
// BAD - UI freezes
Button("Go") {
    let r = try await session.respond(to: prompt) // Frozen!
}
```
**Fix:** Wrap in `Task {}`.

### Ignoring Context Limits
4096 token limit (input + output). Break large tasks into smaller ones.

## Error Handling

```swift
do {
    let response = try await session.respond(to: prompt)
} catch LanguageModelSession.GenerationError.exceededContextWindowSize {
    // Condense transcript, create new session
    session = condensedSession(from: session)
} catch LanguageModelSession.GenerationError.guardrailViolation {
    showMessage("I can't help with that request")
} catch LanguageModelSession.GenerationError.unsupportedLanguageOrLocale {
    showMessage("Language not supported")
}
```

## Context Overflow Fix

```swift
func condensedSession(from prev: LanguageModelSession) -> LanguageModelSession {
    let entries = prev.transcript.entries
    guard entries.count > 2 else { return prev }

    // Keep first (instructions) + last (recent)
    let condensed = [entries.first!, entries.last!]
    return LanguageModelSession(transcript: Transcript(entries: condensed))
}
```

## Availability Issues

```swift
switch SystemLanguageModel.default.availability {
case .available:
    // Proceed
case .unavailable:
    // Show: "AI requires iPhone 15 Pro+ or M1 iPad/Mac"
    // Or: "Enable in Settings > Apple Intelligence"
}
```

## Performance Fixes

| Issue | Solution |
|-------|----------|
| Slow first generation | Prewarm session in `init()` |
| Long wait for results | Use `streamResponse()` |
| Schema overhead | `includeSchemaInPrompt: false` on subsequent requests |
| Complex prompt | Break into multiple smaller calls |

## Quick Reference

| Symptom | Cause | Fix |
|---------|-------|-----|
| Won't start | .unavailable | Check device/region/opt-in |
| exceededContextWindowSize | >4096 tokens | Condense transcript |
| guardrailViolation | Content policy | Handle gracefully |
| Hallucinated output | Wrong use case | Use for extraction only |
| Wrong structure | No @Generable | Use @Generable |
| Initial delay | Model loading | Prewarm session |
| UI frozen | Main thread | Use Task {} |

## Checklist

- [ ] Availability checked before session
- [ ] Using @Generable (not JSON)
- [ ] Handling context overflow
- [ ] Handling guardrails
- [ ] Streaming for >1s generations
- [ ] Not blocking UI
- [ ] Tools for external data
- [ ] Not using for world knowledge

```