README

Xberg

liter-llm

One API for every LLM — in your language, without the rewrites.

What and Why?

liter-llm gives you a single, consistent way to call any large language model. Reach 165 providers — OpenAI, Anthropic, Google, Bedrock, and more — through one client, and switch models by changing a name instead of your code. The same client ships to 14 languages, all built on one Rust core, so you get identical behavior everywhere. Need a gateway? Point your existing OpenAI SDK at the built-in, drop-in proxy — no rewrites, no separate service to run.

Work with any provider without rewrites — 165 providers behind one API; change the model name to switch, your code stays the same.
Use it from your language — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all built on the same Rust core, plus a C/FFI surface for everything else.
Secure by default — API keys are wrapped, redacted from logs, and never serialized; cloud auth for Azure, AWS, and Vertex refreshes and rotates for you.
One small binary — the client, a drop-in OpenAI-compatible proxy, and an MCP server for AI agents ship in a single 35 MB binary with no interpreter and no dependency tree to compromise.
Stay up when a provider fails — route by cost, latency, or weight and fall back automatically, with built-in rate limiting, caching, and cost tracking.

Deeper details — observability, middleware, and the full endpoint list — live in the documentation.

We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.

Features

Feature	Description
165 providers	OpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time
14 native bindings	Rust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface
First-class streaming	SSE and AWS EventStream binary protocol with zero-copy buffers
Proxy & MCP server	Drop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image
Tower middleware	Rate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable
Observability	OpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing
Tool calling	Parallel tools, structured outputs, and JSON-schema validation
Search & OCR	Web search across 12 providers, document OCR across 4
TOML configuration	`liter-llm.toml` auto-discovery, custom providers, cache backends, and middleware config
Local LLM support	Ollama, LM Studio, vLLM, llama.cpp, LocalAI, and llamafile via OpenAI-compatible APIs

⭐ Star this repo to show your support — it helps others discover liter-llm.

Quick Start

Language Packages

Python

pip install liter-llm

See Python README for full documentation.

Node.js

pnpm add @xberg-io/liter-llm

See Node.js README for full documentation.

Rust

cargo add liter-llm

See Rust crate for full documentation.

go get github.com/xberg-io/liter-llm/packages/go/v2

See Go README for full documentation.

Java

Available on Maven Central as io.xberg.literllm:liter-llm. See Java README for the dependency snippet and current version.

Ruby

gem install liter_llm

See Ruby README for full documentation.

PHP

composer require xberg-io/liter-llm

See PHP README for full documentation.

dotnet add package LiterLlm

See .NET README for full documentation.

Elixir

Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.

Dart / Flutter

dart pub add liter_llm

See Dart README for full documentation.

Swift

Add via Swift Package Manager. See Swift README for full documentation.

Kotlin (Android)

Available on Maven Central as io.xberg.literllm:liter-llm-android. See Kotlin README for the dependency snippet and current version.

Zig

See Zig README for installation and usage.

WebAssembly

pnpm add @xberg-io/liter-llm-wasm

See WebAssembly README for full documentation.

C/C++ (FFI)

Build from source as part of this workspace. See FFI crate for full documentation.

CLI, Proxy & MCP Server

The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:

brew install xberg-io/tap/liter-llm
cargo install liter-llm-cli                 # from crates.io
cargo binstall liter-llm-cli               # prebuilt binary from GitHub Releases
npx @xberg-io/liter-llm-cli --help         # npm (self-installs the binary)
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm

Windows users can install the same binary through Scoop:

scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install liter-llm

Then run the proxy or the MCP server:

liter-llm api --config liter-llm-proxy.toml   # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio               # MCP tool server (stdio)
liter-llm mcp --transport http --port 3001    # MCP tool server (Streamable HTTP)

See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it auto-registers the server, no manual config required.

AI Coding Assistants

Install the liter-llm plugin from xberg-io/liter-llm. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 165 providers) and works with every major coding agent — expand your harness below.

Claude Code

/plugin marketplace add xberg-io/liter-llm
/plugin install liter-llm@liter-llm

Codex CLI

/plugins add https://github.com/xberg-io/liter-llm

Then search for liter-llm and select Install Plugin.

Cursor

Settings → Plugins → Add from URL → https://github.com/xberg-io/liter-llm, then select liter-llm.

Gemini CLI

gemini extensions install https://github.com/xberg-io/liter-llm

Factory Droid

droid plugin marketplace add https://github.com/xberg-io/liter-llm
droid plugin install liter-llm@liter-llm

GitHub Copilot CLI

copilot plugin marketplace add https://github.com/xberg-io/liter-llm
copilot plugin install liter-llm@liter-llm

opencode

Add the package to opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@xberg-io/opencode-liter-llm"]
}

Documentation

Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Join our Discord community for questions and discussion.

Part of Xberg.io

Xberg — document intelligence: text, tables, metadata from 101 formats with optional OCR.
Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
html-to-markdown — fast, lossless HTML→Markdown engine.
liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

License

MIT — see LICENSE for details.

xberg-io / liter-llm

Package info

Security

Statistics