README

Transactional outbox for PHP, and a small relay in Go that publishes the events to Kafka or RabbitMQ. No Debezium, no Kafka Connect.

Status: 1.0, stable. What a version promises is in ADR 0008: a change to it means 2.0. Bug reports and "this did not fit my setup" are welcome in issues. PostgreSQL or MySQL, and Kafka or RabbitMQ.

The problem

A service saves an order and then sends OrderPlaced to Kafka. If the process dies between the commit and the send, the order exists and nobody downstream hears about it. Send first and crash before the commit, and consumers react to an order that was never saved. No transaction spans both the database and the broker.

The outbox pattern fixes this with one extra table. The event goes into outbox in the same transaction as the order, so either both are saved or neither is. A separate process reads the table and publishes to the broker.

For that second half, PHP teams usually pick between Debezium with Kafka Connect (a whole platform to run for a team of five) and a polling loop written again in every project. This repository is the polling loop, written once, with the parts that are easy to get wrong: order per aggregate, several replicas, failover, metrics.

What is in it

ianfoxdev/outbox, a Composer package. Outbox::record() inserts the event into the outbox table through the connection you already write with: PDO, Doctrine DBAL 3.8+, or Laravel's, on PostgreSQL or MySQL. It refuses to run outside a transaction. A Laravel service provider and a Symfony bundle wire it up and create the table; with Doctrine ORM, entities can record their own events and the flush writes them. For unit tests, InMemoryRecorder stands in for it behind the Recorder interface.
outbox-relay, a static Go binary in a 33 MB distroless image. It publishes each row as a Kafka record or a RabbitMQ message with CloudEvents attributes, keeps the events of one aggregate in order, lets several replicas run with one of them publishing, deletes old rows, and exposes Prometheus metrics.

Quick start

Laravel 12 or 13

composer require ianfoxdev/outbox
php artisan vendor:publish --tag=outbox-migrations
php artisan migrate

use IanFoxDev\Outbox\Message;
use IanFoxDev\Outbox\Outbox;

DB::transaction(function () use ($order, $outbox) {
    $order->status = 'placed';
    $order->save();

    $outbox->record(Message::json('OrderPlaced', 'order', $order->id, [
        'total' => $order->total,
    ]));
});

Symfony 7.4 or 8

Enable IanFoxDev\Outbox\Bridge\Symfony\OutboxBundle, set outbox.source (for example /orders), then:

bin/console outbox:migration
bin/console doctrine:migrations:migrate

Inject Outbox and call record() inside wrapInTransaction() or Connection::transactional(). More in docs/symfony.md.

Plain PDO

$outbox = new Outbox(new PdoConnection($pdo), source: '/orders');

$pdo->beginTransaction();
$pdo->prepare('UPDATE orders SET status = ? WHERE id = ?')->execute(['placed', 42]);
$outbox->record(Message::json('OrderPlaced', 'order', 42, ['total' => 1999]));
$pdo->commit();

The table is in schema/postgresql.sql and schema/mysql.sql.

The relay

docker run -p 8080:8080 \
  -e OUTBOX_DATABASE_URL=postgres://app:secret@db:5432/app \
  -e OUTBOX_KAFKA_BROKERS=kafka:9092 \
  ghcr.io/ianfoxdev/outbox-relay:1.0

For MySQL, use OUTBOX_DATABASE_URL=mysql://app:secret@db:3306/app. An order event goes to the order.events topic, keyed by the order id. Create the topics yourself; the relay does not.

For RabbitMQ, replace the Kafka setting with OUTBOX_PUBLISHER=rabbitmq, OUTBOX_RABBITMQ_URL=amqp://app:secret@rabbitmq:5672/ and OUTBOX_RABBITMQ_EXCHANGE=events. The event then goes to the exchange with the routing key order.OrderPlaced; the exchange and the queues are yours to declare. All settings are in docs/relay.md.

To see everything working at once, docker compose up --build in the repository root starts Postgres, Kafka and two relays (compose.rabbitmq.yaml swaps Kafka for RabbitMQ), and examples/ has a Laravel and a Symfony shop with tests that read the events back from Kafka.

Guarantees and limits

The event is saved if and only if your transaction commits.
Delivery is at-least-once. After a relay failover a batch can be sent twice, with the same ce_id. Consumers deduplicate by it: docs/consuming.md shows how, in the same transaction as their own writes.
Events of one aggregate reach the topic or queue in the order their transactions committed, as long as your code serializes writes to one aggregate (a row lock or a version check taken before record()). If two transactions change one order truly in parallel, there is no order to keep. Why: ADR 0002.
One replica publishes at a time. That keeps the order simple and is fast enough for most services: 44000 to 71000 events/s draining a backlog on a laptop with PostgreSQL and Kafka, 32000 to 46000 with 1 to 2 ms of network delay added, about half that with MySQL. On RabbitMQ the relay waits for confirms to keep the order, and a backlog of one busy aggregate drains at one confirm per event. See docs/benchmarks.md.
A row that cannot be published (for example, its topic name is invalid) holds back the later events of its aggregate until it is fixed or deleted. Other aggregates go on. There is no dead letter queue: it would break the order.
PostgreSQL 16 or later, or MySQL 8.4 or later. MariaDB is not supported. Kafka, or RabbitMQ 4 over AMQP 0-9-1.

A test kills the leader with SIGKILL four times while 20 writers insert about 100000 events; no event is lost and every aggregate stays in order (docs/relay.md).

Documentation

When to use it: what polling costs, when Debezium or another library fits better.
Architecture: table layout, write path, relay loop, what happens on each kind of failure.
Relay: settings, Kafka records and RabbitMQ messages, replicas and the leader lock, metrics and alerts, cleanup, Docker.
Runbook: what to check when an alert fires, SQL for both databases, and what fixing rows by hand costs.
Kubernetes: kustomize base, ServiceMonitor and alerts, and a test on kind.
Laravel and Symfony setup.
Consuming events: headers, duplicates, offsets, order on the consumer side.
Benchmarks: how fast, and what limits it.
Decisions: one repository, a single active relay, CloudEvents headers, franz-go, MySQL, events from Doctrine ORM entities, RabbitMQ, what 1.0 promises.

Requirements

Part	Versions tested in CI
PHP package	PHP 8.3, 8.4, 8.5; PostgreSQL 16, 17, 18; MySQL 8.4 and 9; Doctrine DBAL 3.8 and 4; Laravel 12 and 13; Symfony 7.4 and 8
Relay	PostgreSQL 18, MySQL 8.4, Kafka 4.3, RabbitMQ 4.3; images for linux/amd64 and linux/arm64

Upgrading

From 1.0 the project follows semantic versioning strictly (ADR 0008 lists what that covers, down to the lock key and the partitioning). Before 1.0 a minor version could break the API. UPGRADE.md says which relay goes with which package version and what to change between versions.

Contributing

See CONTRIBUTING.md. Security issues: SECURITY.md.

License

MIT

ianfoxdev / outbox

Package info

Security

Statistics