README

A resilient, provider-agnostic text-generation gateway for Laravel.

Laravel LLM Failover protects chat applications from upstream rate limits, overloads, timeouts, server errors, refusals, and empty responses. It tries each configured provider in order and returns one normalized response shape, regardless of which provider succeeds.

Features

Ordered failover chains across Gemini, OpenAI, and Anthropic
Configurable retries, delay, connection timeout, and request timeout
Graceful, configurable reply after the entire chain is exhausted
Consistent response and token-usage schema for every provider
Normalized roles and collapsed consecutive same-role messages
Eloquent conversation-history builder with deterministic tie-breaking
Custom drivers through Llm::extend()
Laravel container binding, facade, package discovery, and publishable config
Typed exceptions without API keys or raw provider bodies in normal error messages
PHPUnit, Orchestra Testbench, level-8 PHPStan, dependency audit, and CI coverage
Runnable Postman examples with credential guards and response assertions

Requirements

PHP 8.2 or newer
Laravel 10, 11, 12, or 13

Laravel 13 itself requires PHP 8.3 or newer.

Laravel 10 and 11 are retained for backwards compatibility but no longer receive upstream security fixes. Use a currently supported Laravel release for production; the CI dependency audit runs on Laravel 12 and 13, while the older lines receive compatibility tests only.

Installation

composer require tawshiqulislam/laravel-llm-failover

Publish the configuration:

php artisan vendor:publish --tag="llm-failover-config"

This creates config/llm-failover.php. Laravel package discovery registers the service provider and the Llm facade automatically.

Configuration

Single provider

LLM_DEFAULT_DRIVER=gemini
LLM_FAILOVER_ENABLED=false

GEMINI_API_KEY=your_actual_gemini_api_key

When failover is disabled, LlmDriverInterface resolves directly to LLM_DEFAULT_DRIVER.

Production failover

LLM_DEFAULT_DRIVER=gemini
LLM_FAILOVER_ENABLED=true
LLM_FAILOVER_CHAIN=gemini,openai,anthropic

GEMINI_API_KEY=your_gemini_key
OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key

Provider names are trimmed, lowercased, and deduplicated. An empty chain uses LLM_DEFAULT_DRIVER.

All environment options

# Failover
LLM_DEFAULT_DRIVER=gemini
LLM_FAILOVER_ENABLED=true
LLM_FAILOVER_CHAIN=gemini,openai,anthropic
LLM_FALLBACK_REPLY="Sorry, I am having trouble responding right now. Please try again shortly."
LLM_LOG_FALLBACKS=true

# Shared HTTP behavior
LLM_HTTP_TIMEOUT=30
LLM_HTTP_CONNECT_TIMEOUT=10
LLM_HTTP_RETRIES=2
LLM_HTTP_RETRY_DELAY=250

# Gemini
GEMINI_API_KEY=
GEMINI_DEFAULT_MODEL=gemini-2.5-flash
GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/models
GEMINI_MAX_OUTPUT_TOKENS=1000
GEMINI_TEMPERATURE=0.7

# OpenAI
OPENAI_API_KEY=
OPENAI_DEFAULT_MODEL=gpt-4o-mini
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MAX_OUTPUT_TOKENS=1000
OPENAI_TEMPERATURE=0.7

# Anthropic
ANTHROPIC_API_KEY=
ANTHROPIC_DEFAULT_MODEL=claude-opus-4-8
ANTHROPIC_BASE_URL=https://api.anthropic.com/v1
ANTHROPIC_VERSION=2023-06-01
ANTHROPIC_MAX_OUTPUT_TOKENS=1024
# Omitted by default because current Claude models may reject it.
ANTHROPIC_TEMPERATURE=null

LLM_HTTP_RETRIES is the number of retries after the initial attempt. A value of 2 allows at most three HTTP attempts per provider. Connection failures, 408, 429, every 5xx, and Anthropic 529 responses are retryable.

Gemini and OpenAI default to a temperature of 0.7. Set the relevant value to the literal null if the selected model does not accept sampling parameters:

GEMINI_TEMPERATURE=null
OPENAI_TEMPERATURE=null
ANTHROPIC_TEMPERATURE=null

Each driver config array also accepts timeout, connect_timeout, retries, and retry_delay; driver-specific values override the shared HTTP settings:

'drivers' => [
    'openai' => [
        // ...
        'timeout' => 15,
        'retries' => 1,
    ],
],

After changing .env values in a config-cached application, rebuild the cache:

php artisan config:cache

Usage

Dependency injection

Injecting the interface is the recommended application-facing entry point. It resolves to the failover gateway when failover is enabled and to the default driver otherwise.

use TawshiqulIslam\LlmFailover\Contracts\LlmDriverInterface;

final class ChatResponseService
{
    public function __construct(
        private LlmDriverInterface $aiClient,
    ) {}

    public function processReply(string $systemPrompt, array $history): string
    {
        // An empty model string uses the active driver's configured default.
        $result = $this->aiClient->send('', $systemPrompt, $history);

        if ($result['is_fallback']) {
            logger()->warning($result['error']);
        }

        return $result['reply'];
    }
}

Conversation history must be chronological and contain at least one non-empty text message:

$history = [
    ['role' => 'user', 'content' => 'Hello!'],
    ['role' => 'assistant', 'content' => 'How can I help?'],
    ['role' => 'user', 'content' => 'When do you open?'],
];

Recognized assistant aliases are assistant, model, ai, and bot. Other role values normalize to user. Blank messages are removed, and consecutive messages with the same normalized role are joined with a newline.

Facade

use TawshiqulIslam\LlmFailover\Facades\Llm;

// Walk the configured failover chain.
$result = Llm::send('', 'You are a helpful assistant.', $history);

// Call one provider directly without cross-provider failover.
$result = Llm::driver('anthropic')->send('', 'Be concise.', $history);

Model selection during failover

Only the first provider receives the model passed to send(). A different provider cannot generally use that model name, so every later provider uses its own configured default.

With LLM_FAILOVER_CHAIN=gemini,openai:

Gemini receives the caller's model, or GEMINI_DEFAULT_MODEL when the model is empty.
Gemini applies its configured retry policy to retryable failures.
If Gemini still fails, OpenAI is called with OPENAI_DEFAULT_MODEL.
If every provider fails, the gateway returns LLM_FALLBACK_REPLY.

Retryable responses, network failures, empty replies, safety blocks, and supported refusal responses cause failover. Missing API keys and non-retryable provider rejections are also skipped inside the gateway so another configured provider can answer.

Building Eloquent conversation history

Use HasChronologicalHistory on the model that owns a HasMany message relation:

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Relations\HasMany;
use TawshiqulIslam\LlmFailover\Traits\HasChronologicalHistory;

final class Conversation extends Model
{
    use HasChronologicalHistory;

    public function messages(): HasMany
    {
        return $this->hasMany(Message::class);
    }
}

Fetch the newest 20 messages and return them oldest-first:

$history = $conversation->getChronologicalHistory(
    $conversation->messages(),
    20,
);

The message limit must be at least 1. By default, the trait orders by the related model's id. It reads text from body, then content, and maps either of these common schemas:

Stored field	User values	Assistant values
`direction`	`inbound`	Any other non-null direction, normally `outbound`
`role`	`user`, `inbound`, `human`, `customer`	Any other value, normally `assistant`

When both fields exist, direction takes precedence.

For UUID primary keys, pass a monotonically increasing sequence column whenever possible:

$history = $conversation->getChronologicalHistory(
    $conversation->messages(),
    20,
    'position',
);

The related primary key is used as a stable secondary sort when the chosen order column contains ties. That makes selection deterministic, but a random UUID cannot reconstruct the true creation order of rows with identical timestamps; use a sequence column if exact ordering matters.

Custom drivers

use TawshiqulIslam\LlmFailover\Facades\Llm;
use TawshiqulIslam\LlmFailover\LlmManager;
use App\Llm\MistralDriver;

Llm::extend('mistral', function ($app) {
    return new MistralDriver(
        $app->make(LlmManager::class)->driverConfig('mistral'),
    );
});

Add the driver's config under llm-failover.drivers.mistral, then include mistral in LLM_FAILOVER_CHAIN. A custom driver must implement LlmDriverInterface and return the response shape below. Extending AbstractDriver provides the shared history normalization, HTTP retry configuration, response helpers, and fallback logging.

Response format

Every driver and the gateway implement the same contract:

[
    'reply' => 'Generated response',
    'is_fallback' => false,
    'error' => null,
    'usage' => [
        'input_tokens' => 124,
        'output_tokens' => 45,
        'cached_input_tokens' => 0,
    ],
    'model' => 'gemini-2.5-flash',
    'driver' => 'gemini',
]

Field	Type	Description
`reply`	`string`	Provider text, or the configured graceful fallback reply.
`is_fallback`	`bool`	`true` when no provider text was returned.
`error`	`string\|null`	A safe provider or aggregated failure description.
`usage.input_tokens`	`int`	Input tokens reported by the successful provider.
`usage.output_tokens`	`int`	Output tokens reported by the successful provider.
`usage.cached_input_tokens`	`int`	Cached input tokens reported by the provider.
`model`	`string`	The requested/configured model associated with the result.
`driver`	`string\|null`	Successful/direct driver name; `null` when the gateway exhausts the chain.

Fallback usage values are zero because not every failed provider returns comparable usage metadata. See Expected_Response_Schema.md for the focused schema reference.

Error behavior

Direct provider calls and gateway calls intentionally differ:

Condition	Direct `Llm::driver(...)->send()`	Gateway / injected interface
Network error, retryable status, empty reply, refusal	Returns provider fallback	Tries the next provider
Missing API key	Throws `MissingApiKeyException`	Tries the next provider
Non-retryable provider rejection	Throws `ProviderRequestException`	Tries the next provider
Unknown driver in the configured chain	Throws `UnsupportedDriverException`	Throws immediately
Empty or malformed conversation history	Throws `InvalidArgumentException`	Throws immediately
Every configured provider fails	Not applicable	Returns aggregate fallback

ProviderRequestException exposes driver(), status(), and responseBody(). Its normal exception message deliberately excludes the raw response body; inspect responseBody() explicitly and avoid sending it to end users or untrusted logs.

MissingApiKeyException and ProviderRequestException extend LlmFailoverException, which extends RuntimeException. UnsupportedDriverException extends InvalidArgumentException for backwards compatibility.

Provider APIs

The bundled drivers currently target:

Driver	API
Gemini	`models.generateContent`
OpenAI	`POST /v1/chat/completions`
Anthropic	`POST /v1/messages`

This v1 package supports text conversations only. Streaming, tool calls, images/audio, structured-output helpers, embeddings, and provider-native server-side fallbacks are outside the current interface.

Postman collection

Import postman/laravel-llm-failover.postman_collection.json into Postman, set one or more API-key collection variables, and run the matching requests. Placeholder-key guards stop a request before it is sent, and each request includes response assertions.

The collection calls providers directly. It validates provider credentials and payload compatibility, but it does not run Laravel or test the package's failover chain. See POSTMAN_COLLECTION.md for detailed instructions and Newman usage.

Never commit real API keys to the collection or an exported Postman environment. Live runs may incur provider charges.

Testing and quality checks

# PHPUnit only
composer test

# PHPUnit plus level-8 PHPStan
composer check

# Dependency advisories
composer audit

The PHPUnit suite uses Laravel HTTP fakes and SQLite in memory; no API credentials or network access are required. CI covers the compatible Laravel/PHP combinations and runs Composer validation, tests, and static analysis. Dependency auditing runs on the currently security-supported Laravel 12 and 13 jobs.

Security

API keys are sent in headers and are never placed in provider URLs.
Raw provider response bodies are available only through the explicit exception accessor.
Fallback logs contain safe messages and status metadata, not raw response bodies.
Disable fallback logging with LLM_LOG_FALLBACKS=false if even provider/status metadata is unsuitable for your environment.

Please report security issues privately to the maintainer email listed in composer.json rather than opening a public issue.

Contributing

Contributions are welcome. Please include tests and update the relevant documentation when behavior changes.

Useful contribution areas include additional providers, failover strategies, richer response types, and performance improvements. Run composer check before submitting a pull request.

License

The MIT License. See LICENSE.md.

tawshiqulislam / laravel-llm-failover

Maintainers

Package info

Statistics

Security

README

Features

Requirements

Installation

Configuration

Single provider

Production failover

All environment options

Usage

Dependency injection

Facade

Model selection during failover

Building Eloquent conversation history

Custom drivers

Response format

Error behavior

Provider APIs

Postman collection

Testing and quality checks

Security

Contributing

License