Skip to content

AI SDK Provider

The ai-sdk-provider-saturn package integrates Saturn discovery into the Vercel AI SDK for TypeScript and JavaScript applications.

Installation

npm install ai-sdk-provider-saturn

Circuit Breaker

Closed (requests flow) 3 failures Open (requests blocked) 30s cooldown Half-Open (probe request) success failure

Features

  • Implements Vercel AI SDK ProviderV3 interface
  • Uses multicast-dns npm library for mDNS discovery
  • Circuit breaker: threshold 3 failures, reset after 30s cooldown
  • Direct mode: bypass mDNS, connect to a known endpoint
  • Dynamic model registration/unregistration via event callbacks
  • Auto-failover between services

Usage

import { createSaturnProvider } from 'ai-sdk-provider-saturn'
import { generateText } from 'ai'

const saturn = createSaturnProvider({ timeout: 5000 })

const { text } = await generateText({
  model: saturn('service/model'),
  prompt: 'Hello'
})

The provider discovers Saturn services on the network, resolves their models, and routes requests to the best available backend. If a service fails, the circuit breaker opens and requests fail over to the next service by priority.

Direct Mode

Skip mDNS discovery and connect to a known endpoint:

const saturn = createSaturnProvider({
  direct: { baseURL: 'http://localhost:11434/v1' }
})

Event Callbacks

React to services appearing and disappearing on the network:

const saturn = createSaturnProvider({
  timeout: 5000,
  onServiceFound: (service) => console.log(`Found: ${service.name}`),
  onServiceLost: (service) => console.log(`Lost: ${service.name}`)
})

Model References

Models are referenced as service/model:

saturn('ollama/llama3.2')
saturn('openrouter/anthropic/claude-3.5-sonnet')

If no service is specified, the provider selects the best available service that offers the requested model.