AI Models
The Heroku Managed Inference and Agent add-on supports the following models. The add-on is hosted in two regions: us and eu. However, the add-on can be provisioned and accessed from apps in any Heroku region. Select a model to view information on rate limits, prompt caching, and implementation.
| Model Documentation | Region | Type | API Endpoint | Model Source | Description |
|---|---|---|---|---|---|
| Claude 4.5 Sonnet | US, EU | text → text |
/v1/chat/completions | Anthropic | A state-of-the-art LLM optimized for enterprise apps that supports chat, tool-calling, and enhanced reasoning. |
| Claude 4.5 Haiku | US, EU | text → text |
/v1/chat/completions | Anthropic | A state-of-the-art LLM that supports chat, tool-calling, and enhanced reasoning. |
| Claude 4 Sonnet | US, EU | text → text |
/v1/chat/completions | Anthropic | An intelligent and detail-oriented LLM that supports chat, tool-calling, and enhanced reasoning. |
| Claude 3.7 Sonnet | US, EU | text → text |
/v1/chat/completions | Anthropic | An intelligent and detail-oriented LLM that supports chat, tool-calling, and enhanced reasoning. |
| Claude 3.5 Sonnet Latest | US, EU | text → text |
/v1/chat/completions | Anthropic | A fast and affordable LLM that supports chat and tool-calling. |
| Claude 3.5 Haiku | US, EU | text → text |
/v1/chat/completions | Anthropic | An affordable and straightforward LLM that supports chat and tool-calling. |
| Claude 3 Haiku | EU | text → text |
/v1/chat/completions | Anthropic | A fast and affordable LLM that supports chat and tool-calling. |
| Amazon Nova Lite | US, EU | text → text |
/v1/chat/completions | Amazon | A fast and cost-effective LLM. |
| Amazon Nova Pro | US, EU | text → text |
/v1/chat/completions | Amazon | A high-performance LLM designed for complex tasks. |
| OpenAI gpt-oss-120b | US, EU | text → text |
/v1/chat/completions | OpenAI | An open-weight LLM that supports chat and tool-calling. |
| Cohere Embed Multilingual | US, EU | text → embedding |
/v1/embeddings | Cohere | A state-of-the-art embedding model that supports multiple languages and can be helpful for developing RAG (Retrieval Augmented Generation) search. |
| Stable Image Ultra | US, EU | text → image |
/v1/images/generations | Stability AI | A state-of-the-art diffusion (image generation) model. |