Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

Use it as a **Python SDK** for direct library integration, or deploy the **AI Gateway (Proxy Server)** as a centralized service for your team or organization.

**Jump to LiteLLM Proxy (LLM Gateway) Docs**

Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:

- **Unified API** — one interface for 100+ LLMs, no provider-specific SDK juggling - **Drop-in OpenAI compatibility** — swap providers without rewriting your code - **Production-ready gateway** — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box - **8ms P95 latency** at 1k RPS (benchmarks)

**All Supported Endpoints** - `/chat/completions`, `/responses`, `/embeddings`, `/images`, `/audio`, `/batches`, `/rerank`, `/a2a`, `/messages` and more.

`uv add litellm` An independent `litellm-core` distribution provides the Python SDK with the same `import litellm` API and runtime dependencies. It has no optional extras, CLI entry points, or bundled dashboard. Install one SDK distribution per environment because `litellm` and `litellm-core` own overlapping Python files. Use `litellm` for the proxy, CLI, and optional extras

Build and install core from this checkout while its release integration is pending:

``` python scripts/build_core_distribution.py --out-dir dist/core python -m pip install dist/core/litellm_core-*.whl ``` The builder requires Git, `uv` and the Rust build toolchain. It reads the release version from the root `pyproject.toml`, leaves source files unchanged, and produces a wheel and a self-contained sdist in `dist/core`. Run installation in a fresh environment without `litellm`

``` from litellm import completion import os os.environ["OPENAI_API_KEY"] = "your-openai-key" os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key" # OpenAI response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}]) # Anthropic response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}]) ``` **Getting Started - E2E Tutorial** - Setup virtual keys, make your first request

``` uv tool install 'litellm[proxy]' litellm --model gpt-4o ``` ``` import openai client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000") response = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}] ) ``` ## **Agents** - Invoke A2A Agents (Python SDK + AI Gateway)

**Supported Providers** - LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, Pydantic AI

``` from litellm.a2a_protocol import A2AClient from a2a.types import SendMessageRequest, MessageSendParams from uuid import uuid4 client = A2AClient(base_url="http://localhost:10001") request = SendMessageRequest( id=str(uuid4()), params=MessageSendParams( message={ "role": "user", "parts": [{"kind": "text", "text": "Hello!"}], "messageId": uuid4().hex, } ) ) response = await client.send_message(request) ``` **Step 1.** Add your Agent to the AI Gateway — set `protocolVersion` to `1.0` or `0.3` per agent

**Step 2.** Call Agent via A2A SDK (requires `a2a-sdk>=1.1.0`)