> ## Documentation Index
> Fetch the complete documentation index at: https://docs.elizaos.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Language Model Configuration

> Understanding and configuring Language Model plugins in elizaOS

elizaOS uses a plugin-based architecture for integrating different Language Model providers. This guide explains how to configure and use LLM plugins, including fallback mechanisms for embeddings and model registration.

## Key Concepts

### Model Types

elizaOS supports many types of AI operations. Here are the most common ones:

1. **TEXT\_GENERATION** (`TEXT_SMALL`, `TEXT_LARGE`) - Having conversations and generating responses
2. **TEXT\_EMBEDDING** - Converting text into numbers for memory and search
3. **OBJECT\_GENERATION** (`OBJECT_SMALL`, `OBJECT_LARGE`) - Creating structured data like JSON

Think of it like different tools in a toolbox:

* **Text Generation** = Having a conversation
* **Embeddings** = Creating a "fingerprint" of text for finding similar things later
* **Object Generation** = Filling out forms with specific information

### Plugin Capabilities

Not all LLM plugins support all model types. Here's what each can do:

| Plugin       | Text Chat | Embeddings | Structured Output | Runs Offline |
| ------------ | --------- | ---------- | ----------------- | ------------ |
| OpenAI       | ✅         | ✅          | ✅                 | ❌            |
| Anthropic    | ✅         | ❌          | ✅                 | ❌            |
| Google GenAI | ✅         | ✅          | ✅                 | ❌            |
| Ollama       | ✅         | ✅          | ✅                 | ✅            |
| OpenRouter   | ✅         | ✅          | ✅                 | ❌            |

**Key Points:**

* 🌟 **OpenAI, Google GenAI & OpenRouter** = Do everything (jack of all trades)
* 💬 **Anthropic** = Amazing at chat, needs a fallback for embeddings
* 🏠 **Ollama** = Your local hero - does almost everything, no internet needed!

## Plugin Loading Order

The order in which plugins are loaded matters significantly. From the default character configuration:

```typescript theme={null}
plugins: [
  // Core plugins first
  '@elizaos/plugin-sql',

  // Text-only plugins (no embedding support)
  ...(process.env.ANTHROPIC_API_KEY?.trim() ? ['@elizaos/plugin-anthropic'] : []),

  // Embedding-capable plugins
  ...(process.env.OPENROUTER_API_KEY?.trim() ? ['@elizaos/plugin-openrouter'] : []),
  ...(process.env.OPENAI_API_KEY?.trim() ? ['@elizaos/plugin-openai'] : []),
  ...(process.env.GOOGLE_GENERATIVE_AI_API_KEY?.trim() ? ['@elizaos/plugin-google-genai'] : []),

  // Ollama as fallback (only if no main LLM providers are configured)
  ...(process.env.OLLAMA_API_ENDPOINT?.trim() ? ['@elizaos/plugin-ollama'] : []),
]
```

### Understanding the Order

Think of it like choosing team players - you pick specialists first, then all-rounders:

1. **Anthropic goes first** - It's a specialist! Great at text generation but can't do embeddings. By loading it first, it gets priority for text tasks.

2. **OpenRouter, OpenAI & Google GenAI come next** - These are the all-rounders! They can do everything: text generation, embeddings, and structured output. They act as fallbacks for what Anthropic can't do (embeddings).

3. **Ollama comes last** - This is your local backup player! It supports almost everything (text, embeddings, objects) and runs on your computer. Perfect when cloud services aren't available.

### Why This Order Matters

When you ask elizaOS to do something, it looks for the best model in order:

* **Generate text?** → Anthropic gets first shot (if loaded)
* **Create embeddings?** → Anthropic can't, so OpenAI steps in
* **No cloud API keys?** → Ollama handles everything locally

This smart ordering means:

* You get the best specialized models for each task
* You always have fallbacks for missing capabilities
* You can run fully offline with Ollama if needed

### Real Example: How It Works

Let's say you have Anthropic + OpenAI configured:

```
Task: "Generate a response"
1. Anthropic: "I got this!" ✅ (Priority 100 for text)
2. OpenAI: "I'm here if needed" (Priority 50)

Task: "Create embeddings for memory"
1. Anthropic: "Sorry, can't do that" ❌
2. OpenAI: "No problem, I'll handle it!" ✅

Task: "Generate structured JSON"
1. Anthropic: "I can do this!" ✅ (Priority 100 for objects)
2. OpenAI: "Standing by" (Priority 50)
```

## Model Registration

When plugins load, they "register" what they can do. It's like signing up for different jobs:

```typescript theme={null}
// Each plugin says "I can do this!"
runtime.registerModel(
  ModelType.TEXT_LARGE,        // What type of work
  generateText,                // How to do it
  'anthropic',                 // Who's doing it
  100                         // Priority (higher = goes first)
);
```

### How elizaOS Picks the Right Model

When you ask elizaOS to do something, it:

1. **Checks what type of work it is** (text? embeddings? objects?)
2. **Looks at who signed up** for that job
3. **Picks based on priority** (higher number goes first)
4. **If tied, first registered wins**

**Example**: You ask for text generation

* Anthropic registered with priority 100 ✅ (wins!)
* OpenAI registered with priority 50
* Ollama registered with priority 10

But for embeddings:

* Anthropic didn't register ❌ (can't do it)
* OpenAI registered with priority 50 ✅ (wins!)
* Ollama registered with priority 10

## Embedding Fallback Strategy

Remember: Not all plugins can create embeddings! Here's how elizaOS handles this:

**The Problem**:

* You're using Anthropic (great at chat, can't do embeddings)
* But elizaOS needs embeddings for memory and search

**The Solution**:
elizaOS automatically finds another plugin that CAN do embeddings!

```typescript theme={null}
// What happens behind the scenes:
// 1. "I need embeddings!"
// 2. "Can Anthropic do it?" → No ❌
// 3. "Can OpenAI do it?" → Yes ✅
// 4. "OpenAI, you're up!"
```

### Common Patterns

#### Anthropic + OpenAI Fallback

```json theme={null}
{
  "plugins": [
    "@elizaos/plugin-anthropic",  // Primary for text
    "@elizaos/plugin-openai"       // Fallback for embeddings
  ]
}
```

#### OpenRouter Standalone

```json theme={null}
{
  "plugins": [
    "@elizaos/plugin-openrouter"  // Handles text and embeddings
  ]
}
```

#### OpenRouter + Local Fallback

```json theme={null}
{
  "plugins": [
    "@elizaos/plugin-openrouter",  // Cloud text & embeddings
    "@elizaos/plugin-ollama"        // Offline fallback
  ]
}
```

## Configuration

### Environment Variables

Each plugin requires specific environment variables:

````bash theme={null}
# .env file

# OpenAI
OPENAI_API_KEY=sk-...
OPENAI_SMALL_MODEL=gpt-4o-mini          # Optional: any available model
OPENAI_LARGE_MODEL=gpt-4o               # Optional: any available model

# Anthropic  
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_SMALL_MODEL=claude-3-haiku-20240307    # Optional: any Claude model
ANTHROPIC_LARGE_MODEL=claude-3-5-sonnet-latest   # Optional: any Claude model

# Google GenAI
GOOGLE_GENERATIVE_AI_API_KEY=...
GOOGLE_SMALL_MODEL=gemini-2.0-flash-001  # Optional: any Gemini model
GOOGLE_LARGE_MODEL=gemini-2.5-pro-preview-03-25  # Optional: any Gemini model

# Ollama
OLLAMA_API_ENDPOINT=http://localhost:11434/api
OLLAMA_SMALL_MODEL=llama3.2              # Optional: any local model
OLLAMA_LARGE_MODEL=llama3.1:70b          # Optional: any local model
OLLAMA_EMBEDDING_MODEL=nomic-embed-text  # Optional: any embedding model

# OpenRouter
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_SMALL_MODEL=google/gemini-2.0-flash-001  # Optional: any available model
OPENROUTER_LARGE_MODEL=anthropic/claude-3-opus      # Optional: any available model

**Important**: The model names shown are examples. You can use any model available from each provider.

### Character-Specific Secrets

You can also configure API keys per character:

```json
{
  "name": "MyAgent",
  "settings": {
    "secrets": {
      "OPENAI_API_KEY": "sk-...",
      "ANTHROPIC_API_KEY": "sk-ant-..."
    }
  }
}
````

## Available Plugins

### Cloud Providers

* [OpenAI Plugin](/plugin-registry/llm/openai) - Full-featured with all model types
* [Anthropic Plugin](/plugin-registry/llm/anthropic) - Claude models for text generation
* [Google GenAI Plugin](/plugin-registry/llm/google-genai) - Gemini models
* [OpenRouter Plugin](/plugin-registry/llm/openrouter) - Access to multiple providers

### Local/Self-Hosted

* [Ollama Plugin](/plugin-registry/llm/ollama) - Run models locally with Ollama

## Best Practices

### 1. Always Configure Embeddings

Even if your primary model doesn't support embeddings, always include a fallback:

```json theme={null}
{
  "plugins": [
    "@elizaos/plugin-anthropic",
    "@elizaos/plugin-openai"  // For embeddings
  ]
}
```

### 2. Order Matters

Place your preferred providers first, but ensure embedding capability somewhere in the chain.

### 3. Test Your Configuration

Verify all model types work:

```typescript theme={null}
// The runtime will log which provider is used for each operation
[AgentRuntime][MyAgent] Using model TEXT_GENERATION from provider anthropic
[AgentRuntime][MyAgent] Using model EMBEDDING from provider openai
```

### 4. Monitor Costs

Different providers have different pricing. Consider:

* Using local models (Ollama) for development
* Mixing providers (e.g., OpenRouter for text, local for embeddings)
* Setting up usage alerts with your providers

## Troubleshooting

### "No model found for type EMBEDDING"

Your configured plugins don't support embeddings. Add an embedding-capable plugin:

```json theme={null}
{
  "plugins": [
    "@elizaos/plugin-anthropic",
    "@elizaos/plugin-openai"  // Add this
  ]
}
```

### "Missing API Key"

Ensure your environment variables are set:

```bash theme={null}
# Check current environment
echo $OPENAI_API_KEY

# Or use the CLI
elizaos env edit-local
```

### Models Not Loading

Check plugin initialization in logs:

```
Success: Plugin @elizaos/plugin-openai initialized successfully
```

## Migration from v0.x

In elizaOS v0.x, models were configured directly in character files:

```json theme={null}
// ❌ OLD (v0.x) - No longer works
{
  "modelProvider": "openai",
  "model": "gpt-4"
}

// ✅ NEW (v1.x) - Use plugins
{
  "plugins": ["@elizaos/plugin-openai"]
}
```

The `modelProvider` field is now ignored. All model configuration happens through plugins.