fm.cr v0.4.0

Crystal bindings for Apple's FoundationModels framework. Run on-device AI powered by Apple Intelligence directly from Crystal.

Requires macOS 26+ (Tahoe) with Apple Intelligence enabled.

Installation

1. Add the dependency to your shard.yml:

dependencies:
  fm:
    github: hahwul/fm.cr

2. Run shards install

The native Swift FFI library (libfm_ffi.a) is built automatically via postinstall.

Quick Start

require "fm"

model = Fm::SystemLanguageModel.new
model.ensure_available!

session = Fm::Session.new(model, instructions: "You are a helpful assistant.")
response = session.respond("What is the capital of France?")
puts response.content

Features

Basic Conversation

session = Fm::Session.new(model, instructions: "Be concise.")

response = session.respond("What is Crystal?")
puts response.content

# Multi-turn conversation (session maintains context)
response = session.respond("What about its type system?")
puts response.content

Streaming

session = Fm::Session.new(model)

session.stream("Tell me a short story.") do |chunk|
  print chunk
  STDOUT.flush
end
puts

Structured Output

Define a struct with JSON::Serializable and Fm::Generable to get typed responses:

struct Person
  include JSON::Serializable
  include Fm::Generable

  getter name : String
  getter age : Int32
  getter occupation : String
end

person = session.respond_structured(Person, "Generate a fictional software engineer.")
puts "#{person.name}, age #{person.age} — #{person.occupation}"

You can also work with raw JSON schemas directly:

schema = %({"type":"object","properties":{"city":{"type":"string"},"population":{"type":"integer"}},"required":["city","population"]})
json = session.respond_json("Largest city in Japan", schema)
puts json

Schemas are rewritten into the shape FoundationModels' GenerationSchema decoder accepts (title, additionalProperties, x-order, anyOf) before they cross the FFI boundary, so plain JSON Schema works as written and still takes the native guided-generation path.

Generation Guides

Annotate Generable fields with Fm::Guide to constrain what the model may produce:

struct Movie
  include JSON::Serializable
  include Fm::Generable

  @[Fm::Guide(description: "Movie title")]
  getter title : String

  @[Fm::Guide(any_of: ["G", "PG", "PG-13", "R"])]
  getter rating : String

  @[Fm::Guide(minimum: 0, maximum: 10)]
  getter score : Int32

  @[Fm::Guide(pattern: "^[A-Z]")]
  getter director : String

  @[Fm::Guide(min_items: 1, max_items: 5)]
  getter genres : Array(String)
end

Several guides may be stacked on one field; all of them apply, and the last one wins on an option they both set.

Tool Calling

Define tools by subclassing Fm::Tool:

class WeatherTool < Fm::Tool
  def name : String
    "checkWeather"
  end

  def description : String
    "Check current weather conditions for a location"
  end

  def arguments_schema : JSON::Any
    JSON.parse(%({"type":"object","properties":{"location":{"type":"string","description":"City and country"}},"required":["location"]}))
  end

  def call(arguments : JSON::Any) : Fm::ToolOutput
    location = arguments["location"]?.try(&.as_s) || "Unknown"
    Fm::ToolOutput.new("Weather in #{location}: Sunny, 22C")
  end
end

tools = [WeatherTool.new] of Fm::Tool
session = Fm::Session.new(model, instructions: "You have weather capabilities.", tools: tools)

response = session.respond("What's the weather in Tokyo?")
puts response.content

Generation Options

options = Fm::GenerationOptions.new(
  temperature: 0.8,
  sampling: Fm::Sampling::Random,
  max_response_tokens: 500_u32,
  tool_calling_mode: Fm::ToolCallingMode::Allowed # macOS 27+
)

response = session.respond("Write a haiku.", options)
if usage = response.usage
  puts "Used #{usage.total_tokens} tokens"
end

Timeout

response = session.respond("Complex question", timeout: 10.seconds)

Model Availability

model = Fm::SystemLanguageModel.new

case model.availability
when .available?
  puts "Ready"
when .device_not_eligible?
  puts "Device not eligible for Apple Intelligence"
when .apple_intelligence_not_enabled?
  puts "Enable Apple Intelligence in System Settings"
when .model_not_ready?
  puts "Model is downloading..."
end

Token Counting (macOS 26.4+)

if tokens = model.token_count_for("Hello, world!")
  puts "Prompt tokens: #{tokens}"
end

Transcript & Session Restore

# Save conversation state
json = session.transcript_json

# Restore later
restored = Fm::Session.from_transcript(model, json)

Prewarm

session.prewarm("Tell me about")  # hint the model ahead of time

Context Management

Estimate context window usage and compact long conversations:

limit = Fm::ContextLimit.default_on_device(model)
usage = Fm.context_usage_from_transcript(model, session.transcript, limit)

puts "Utilization: #{(usage.utilization * 100).round(1)}%"
puts "Over limit: #{usage.over_limit?}"

# Auto-compact when over limit
if result = Fm.compact_session_if_needed(model, session, limit, base_instructions: "Be helpful.")
  session = result.session
  puts "Compacted. Summary: #{result.summary}"
end

Error Handling

All errors inherit from Fm::Error:

The following inherit from GenerationError and map one-to-one onto the FoundationModels generation error cases:

Every Fm::Error exposes optional .details as parsed JSON. On macOS 27, errors with associated diagnostics also provide typed accessors:

begin
  response = session.respond("Hello")
rescue ex : Fm::ExceededContextWindowSizeError
  puts "#{ex.token_count} tokens exceed the #{ex.context_size}-token context"
rescue ex : Fm::TimeoutError
  puts "Timed out: #{ex.message}"
rescue ex : Fm::ToolCallError
  puts "Tool '#{ex.tool_name}' failed: #{ex.message}"
rescue ex : Fm::Error
  puts "Error: #{ex.message}"
end

API Reference

Fm::SystemLanguageModel

Fm::Session

Fm::GenerationOptions

Fm::SamplingMode adds top-k / top-p control:

Fm::SamplingMode.greedy
Fm::SamplingMode.random(top: 40, seed: 42_u64)              # top-k
Fm::SamplingMode.random(probability_threshold: 0.9)          # top-p (nucleus)

top and probability_threshold are mutually exclusive.

Build Requirements

• macOS 26+ (Tahoe)
• Xcode 26+ with FoundationModels.framework
• Crystal >= 1.21.0
• Swift toolchain (included with Xcode)

Important: The active developer directory must point to the full Xcode installation, not Command Line Tools. See FAQ if you encounter build errors.

FAQ

Build fails with FoundationModelsMacros not found

error: external macro implementation type 'FoundationModelsMacros.GenerableMacro'
could not be found for macro 'Generable(description:)'

This happens when the active developer directory is set to Command Line Tools instead of Xcode. The @Generable macro plugin is only available in the full Xcode installation.

Fix:

sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

You can verify the current setting with:

xcode-select -p
# Should output: /Applications/Xcode.app/Contents/Developer

Model is not available or device not eligible

Apple Intelligence must be enabled on your Mac, and the device must support it (Apple Silicon). Check System Settings > Apple Intelligence & Siri to enable it.

Token counting returns nil

The token_count_for API requires macOS 26.4+ (SDK version 26.4 or later). On older versions, it returns nil by design. The legacy token_usage_for alias behaves the same way.

Contributing

1. Fork it (https://github.com/hahwul/fm.cr/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

License

MIT License. See LICENSE for details.