idct / symfony-nats-messenger
Symfony NATS (JetStream) Messenger Bridge
Package info
github.com/ideaconnect/symfony-nats-messenger
Type:symfony-bundle
pkg:composer/idct/symfony-nats-messenger
Requires
- php: ^8.2
- idct/php-nats-jetstream-client: ^2.29
- psr/log: ^3.0
- symfony/framework-bundle: ^7 || ^8
- symfony/messenger: ^7.3 || ^8.0
- symfony/uid: ^7.2 || ^8.0
Requires (Dev)
- ext-sodium: *
- behat/behat: ^4.0
- dg/bypass-finals: ^1.11
- infection/infection: ^0.32 || ^0.33 || ^0.34 || ^0.35
- phpstan/phpstan: ^2.3
- phpunit/phpunit: ^11.5 || ^12.5 || ^13.4
Suggests
- ext-igbinary: For improved serialization performance when using the default IgbinarySerializer
- ext-sodium: For NKey and JWT authentication: signing the server's nonce with the nkey_seed option needs it
Provides
None
Conflicts
None
Replaces
None
- dev-main
- v5.9.0
- v5.8.1
- v5.8.0
- v5.7.1
- v5.7.0
- v5.6.0
- v5.5.0
- v5.4.1
- v5.4.0
- v5.3.0
- v5.2.3
- v5.2.2
- v5.2.1
- v5.2.0
- v5.1.0
- v5.0.0
- v4.0.0
- v3.2.0
- v3.1.0
- v3.0.0
- v2.0.0
- v1.0.3
- v1.0.2
- v1.0.1
- v1.0.0
- dev-feat/113-retry-via-nak
- dev-chore/release-5.8.1
- dev-bober/issue-106
- dev-chore/release-5.8.0
- dev-docs/record-nkey-and-flaky-merges
- dev-feat/89-nkey-jwt-auth
- dev-fix/flaky-tests
- dev-bober/issue-94
- dev-bober/issue-95
- dev-bober/issue-93
- dev-bober/issue-92
- dev-bober/issue-90
- dev-chore/5.7.1-client-2.27.1
- dev-bober/issue-86
- dev-bober/issue-84
- dev-feat/81-reconnect-mode
- dev-feat/80-stream-placement
- dev-chore/73-74-upgrade-dependencies
- dev-pr-44-fixes
- dev-fix/76-subscription-limit-transport-exception
- dev-chore/require-client-2.10.3
- dev-fix/integer-header-names
- dev-fix/mutation-gate
- dev-fix/dedup-scope-and-idle-pull
- dev-docs/examples-for-new-features
- dev-chore/9-generate-test-certs
- dev-feat/53-duplicate-protection
- dev-fix/54-message-count-dials-once
- dev-docs/55-review-corrections
- dev-chore/52-require-client-2.10
- dev-feat/51-request-timeout
- dev-fix/42-tls-verify-peer-secure-default
- dev-docs/50-retry-delays-need-scheduled-messages
- dev-fix/48-keepalive-does-not-wait
- dev-fix/47-nats-mode-ignores-symfony-retries
- dev-fix/46-strip-non-sendable-stamps
- dev-fix/49-reconnect-after-connection-loss
- dev-pr-40-fixes
- dev-modernize
- dev-new-client
- dev-road_to_stable
This package is auto-updated.
Last update: 2026-11-18 14:44:16 UTC
README
A Symfony Messenger transport integration for NATS JetStream, enabling reliable asynchronous messaging with persistent message streaming.
Features
- π High-Performance Messaging - Leverage NATS JetStream for fast, reliable message delivery
- π¦ Symfony Integration - Implements Messenger's
TransportInterface,MessageCountAwareInterface,SetupableTransportInterface,KeepaliveReceiverInterface, andCloseableTransportInterface - βοΈ Configurable Consumers - Support for multiple consumer strategies
- π Flexible Batching - Adjustable message batch sizes and timeouts
- π Authentication Support - User and password, token, NKey and JWT logins (see NKey Authentication)
- π Stream Configuration - Configurable retention policies and replication
- π Stream Placement - Pin a stream to a JetStream cluster and to tagged servers; an option you set changes only its own half of an existing placement (see Stream Placement)
- π Automatic Reconnect - Optional mode in which the NATS client recovers a lost connection and a worker polls through an outage instead of exiting (see Automatic Reconnect)
- π NATS Clusters - One server in the DSN, the others learned from the cluster; failover, replicated streams and TLS on a cluster (see Using a NATS Cluster)
- β©οΈ Retries by Symfony or by NATS - Symfony's retry strategy carried out as a new message or as a NAK with
Symfony's delay (
retry_via), or NATS's own redelivery (see Retry Handler Behavior) - π§ͺ Thoroughly Tested - 1143 unit tests, ~99.8% coverage, mutation-tested (98.6% MSI)
π This project looks for funding. Love my work? Support it! π
-
β Buy me a coffee: https://buymeacoffee.com/idct
-
π Sponsor: https://github.com/sponsors/ideaconnect
Requirements
System Requirements
- PHP: ^8.2
- Symfony: ^7.3 || ^8
- NATS Server: ^2.9 with JetStream enabled, ^2.12 for scheduled messages support and for
!exclusion tags instream_placement_tags.
Installation
composer require idct/symfony-nats-messenger
The package ships IDCTNatsMessengerBundle, which registers the transport factory and
the igbinary serializer for you - no per-project services.yaml entries are required.
With Symfony Flex the bundle is enabled automatically when the package is installed or updated. Without
Flex, enable it manually:
// config/bundles.php return [ // ... IDCT\NatsMessenger\IDCTNatsMessengerBundle::class => ['all' => true], ];
If you leave the bundle out, register the transport factory yourself. Messenger finds it by its tag, and
nothing else registers it, so without it every nats-jetstream:// DSN fails with "No transport supports the
given Messenger DSN" (before Symfony Messenger 7.4.23 and 8.1.11 the message names the DSN instead):
# config/services.yaml (only without the bundle) services: IDCT\NatsMessenger\NatsTransportFactory: tags: [messenger.transport_factory]
Tested by:
transportFactoryIsRegisteredAndTaggedForMessengerDiscovery,withoutTheBundleNoTransportFactorySupportsANatsDsn,theReadmesIgbinaryExampleGivesTheTransportTheIgbinarySerializer,theReadmesSetupWithoutTheBundleGivesAWorkingTransport- the functional Behat suite also runs entirely on the bundle's service registration.
Upgrading from 5.4 or earlier
- With Symfony Flex, updating the package adds the bundle to
config/bundles.php, as installing it does. Commit that change with the update. - The
services.yamlentries you added for the transport keep working next to the bundle, and you can delete them, with one exception: the bundle does not register theigbinary_serializerid earlier versions of this README suggested. If a transport hasserializer: 'igbinary_serializer', change it toserializer: 'IDCT\NatsMessenger\Serializer\IgbinarySerializer'before you delete that entry, or keep the entry. Otherwise the container cannot be built, because the service the transport names is gone. - Going back to 5.4 or earlier does not take the bundle out of
config/bundles.php. The application then stops withClass "IDCT\NatsMessenger\IDCTNatsMessengerBundle" not found, and so does everycomposer installorcomposer updatethat clears the cache. Delete the line when you downgrade, and add it back yourself if you update again later: Flex configures a package once, and the package's entry insymfony.lockstays after the downgrade.
Tested by:
theServiceEntriesUsedBeforeTheBundleKeepWorking,theReadmesIgbinaryExampleGivesTheTransportTheIgbinarySerializer
Upgrading from 5.6
Nothing to change. The reconnect option is off by default, and with it off the
transport recovers a lost connection as before. Two differences apply in either mode:
- A pull the server ended for its own shutdown (
409 Server Shutdown) or for a leadership change (409 Leadership Change) is an empty batch now, not a missing consumer. Withauto_setupthe transport no longer runssetup()against a server that is going away, which failed every graceful stop of the server with "Failed to setup NATS stream" and ended the worker. Every other 409 keeps its meaning. - If the application has a
loggerservice, the bundle hands it to the NATS client, which writes its connection lifecycle at info level and its errors at error level, under Monolog'smessengerchannel (see Logging). Those lines went nowhere before.
Tested by:
testAPullTheServerEndedForItsShutdownIsEmptyAndTheNextOperationChecksTheConnection,testA409ReportingAConfigurationErrorStaysVisible,theBundleHandsTheApplicationsLoggerToTheFactory, the examplereconnect.php
Upgrading from 5.7.0
Nothing to change. 5.7.1 requires idct/php-nats-jetstream-client 2.27.1 or later, which composer update
installs. With reconnect: true, an ack() or reject() whose write is the first to notice a lost connection
now returns at once, its frame buffered for the connection the client's recovery makes, where with client 2.27.0
it waited for that recovery (see Automatic Reconnect). With the option off nothing
changes.
Tested by:
testAnAcknowledgementWhoseWriteFindsTheConnectionLostReturnsAtOnceAndTheRecoverySendsIt
Upgrading from 5.7.1
Nothing to change in the configuration unless a transport has nkey or jwt set and no nkey_seed, but look at your logs and your retry settings:
nkeyorjwtwith nonkey_seed. Such a transport could never connect, as the client needs the seed to sign the server's nonce, but it could be built. It now fails when it is built, with anInvalidArgumentExceptionnamingnkey_seed. Add the seed or remove the option (see NKey Authentication). Until you do, the earlier error is visible even where nothing uses that transport:messenger:statsshowed 0 for it and exited 0. A plainmessenger:statsnow exits 1 and shows no transport; naming the other transports still works.- A worker consuming from it and from other transports stopped with the client's
requires a nonce signeronce it reached this one: at once if it comes first, otherwise when the transports before it had nothing to do. Under a supervisor it still handled their messages between restarts. It now stops when it starts and handles none. - A
catch (\RuntimeException)around a dispatch to it caught the connection error, and does not catch this one. - A service or smoke test that builds the transport fails.
- A subclass of
NatsTransportthat replaces the protected$clientwith its own signed client must stop passingnkeyorjwthere, as its client has them.
- Two other values are refused now, in the same way: a
max_reconnect_attemptsfloat past the range of an int, such as1e20,1e300or.inf, or a numeric string past the largest float, such as'1e400', which became an unrelated huge number, 0, or a negative number that the client refused, and a credential option after a second?or a;in the DSN query, which was a part of the option before it. So a huge number of attempts is now written as an int or a numeric string up to the largest float, and a value that really holds such text goes in the options, where nothing splits it. - Client 2.28.0 or later. 5.8.0 requires
idct/php-nats-jetstream-client^2.28, whichcomposer updateinstalls. The client now notices, before it writes, that the server has closed the connection while nothing read it, as a server that stops while a handler runs does. Withreconnect: truethe ACK of a message handled at that moment, and Symfony's retry copy of one that failed, go out on the connection the client recovers on another server, where the ACK used to be lost (the message was handled again afterack_wait) and the retry copy timed out and stopped the worker. Withreconnect: falsethat write now fails at once instead of vanishing. See A Handler Failing During the Loss. retry_handler: natswith Symfony's retries on. Up to 5.7.1, on Symfony Messenger 7.4.22 / 8.1.10 or later, a message whose handler failed was lost after its first failure in this setup (#90); 5.8.0 NAKs it, so NATS delivers it again. A transport in nats mode whose retry strategy still retries (FrameworkBundle's defaultmax_retries: 3) now logs a warning once, through yourlogger. Setmax_retries: 0on it (see Retry Handler Behavior).- A failed message costs one round trip more. The NAK or TERM of a message whose handler failed checks the
connection with a PING first, so that it is not lost when the server went away during the handler (see
Losing the Connection).
ping_after_idle: 0turns it off. setup()waits for a consumer NATS is moving after a change ofstream_replicas, for up to 5 seconds and one morerequest_timeout, instead of failing (see High Availability).- New option
resend_on_reconnect(off by default): withreconnect: true, a send with a message id whose server went away before it answered is sent again on the recovered connection (see Automatic Reconnect).
Tested by:
testNatsModeWarnsOnceThatSymfonysRetryStrategyIsOn,testAckNaksADeliveryWhoseRetryCopyNatsModeLeftOut,testSettlingAFailedDeliveryChecksTheConnectionFirst,testSetupAsksForTheConsumerAgainWhileTheClusterIsTemporarilyUnavailable,testTheAckOfAHandledMessageReachesTheServerOnTheRecoveredConnectionWithNoCheckFirst,testSymfonysRetryCopyAfterTheServerClosedTheConnectionDuringTheHandlerGoesOutOnTheRecoveredConnection,testBuildWithResendOnReconnectPropagatesToNatsOptions, the examplesnats-retry.php,handler-failing-during-failover.phpandhandler-succeeding-during-failover.php,testBuildRejectsAnNkeyWithoutASeed,testBuildRejectsAJwtWithoutASeed,testBuildRejectsAMaxReconnectAttemptsThatDoesNotSurviveTheCastToAnInt,testBuildRejectsAMaxReconnectAttemptsInTheDsnThatDoesNotSurviveTheCastToAnInt,testBuildPassesALargeMaxReconnectAttemptsToTheClient,testACredentialTypedAfterASecondQuestionMarkOrASemicolonIsRefusedWithoutRepeatingIt,testAValueThatOnlyLooksLikeACredentialInAQueryIsAccepted, Behat scenarioCredentials that need or exclude a seed are refused when the transport is built, on every command(messenger:setup-transports,messenger:statsand a worker on that transport alone).messenger:statsnaming a transport, and a worker on several transports, were measured by hand.
Upgrading from 5.8.0
- Client 2.29.0 or later. The transport requires
idct/php-nats-jetstream-client^2.29, whichcomposer updateinstalls. ItsdiagnoseNoResponders()tells a pull's 503 from a server that is going away apart from a missing consumer, and its JetStream API calls report a request nothing answers asJetStreamExceptioncode 503. - A pull that gets 503 costs one round trip more: the client asks for the consumer
(
$JS.API.CONSUMER.INFO.<stream>.<consumer>), so that a pull in the last moments of a graceful stop of the server, or during lame duck mode, which gets that 503 for a consumer the server has already stopped, is an empty batch instead of the end of the worker (#106, see Note on a server that stops). A missing consumer still ends a worker withoutauto_setup, as before. For a user allowed to pull but not to ask for consumer info, it ends it only afterrequest_timeout, as the server leaves the question unanswered; allow that subject to have it at once.
Tested by:
testA503KeepsItsReadingWhenTheServerAnswers,testA503KeepsItsReadingWhenNothingAnswersOnAConnectionThatStays,testWithTheRealClientA503FromAServerThatAnswersTheQuestionPropagates,testWithTheRealClientA503WhoseQuestionGoesUnansweredOnAConnectionThatStaysPropagates
Upgrading from 5.8.1
scheduled_messageson a NATS server before 2.12 is refused. It never worked there: the setup succeeded, and every delayed message, Symfony's retry copies among them, was stored and never delivered.messenger:setup-transports, and withauto_setupthe first send or get, now fail withThe 'scheduled_messages' option requires NATS Server >= 2.12, .... Remove the option or upgrade the server. For Symfony's retry delays on such a server, useretry_via: nak.- Symfony Messenger 7.3 or later.
composer.jsonallowed 7.2, on which the transport could not load since 5.0.0. - The warning that Symfony's retry strategy is on in nats mode has a new text, which names
retry_via: nak(see Retry Handler Behavior). A log filter that matches the whole line needs the new one.
Tested by:
testSetupRefusesScheduledMessagesOnAServerWithoutSchedules,testSendRefusesToScheduleOnAServerWithoutSchedules,testNatsModeWarnsOnceThatSymfonysRetryStrategyIsOn, the exampledelayed-messages.php(on the oldest supported server in CI). The unit suite was run by hand on Symfony Messenger 7.3.0, and on 7.2.0 the transport fails to load.
Logging
The NATS client reports what it does with the connection through a PSR-3 logger: the connection's lifecycle
(connected, reconnected, closed) at info level, a reconnect attempt that failed at warning level, and its
errors at error level. Among the errors is the only trace of acknowledgements lost to an outage with
reconnect: true: Reconnect exhausted: N bytes of buffered publishes were discarded.
There is no debug-level chatter, so a wired logger costs a few lines per connection.
The transport writes one line of its own to the same logger: a warning, once per transport object, when a
transport in nats mode leaves out a retry Symfony's retry strategy sent, i.e. when max_retries is above 0 with
retry_handler: nats (see Retry Handler Behavior).
With the bundle, the application's logger service is handed to the factory, tagged for Monolog's
messenger channel, so the client's lines land next to the worker's. Without that service, or without the
bundle, the client gets no logger, as before 5.7.0. To wire one yourself, give it to the factory, which hands
it to every transport it creates, or to a transport constructed directly, as the fourth constructor
argument: new NatsTransport($dsn, $options, $serializer, $logger).
# config/services.yaml (without the bundle, with a logger) services: IDCT\NatsMessenger\NatsTransportFactory: arguments: ['@?logger'] tags: [messenger.transport_factory]
Tested by:
testBuildPassesTheLoggerToTheClientAndNoneByDefault,createTransport_HandsTheFactorysLoggerToTheClient,theBundleHandsTheApplicationsLoggerToTheFactory,theFactoryIsTaggedForMonologsMessengerChannel,testAReconnectThatGivesUpWithABufferedAcknowledgementIsReportedToTheLoggerAndTheNextOperationFails,testNatsModeWarnsOnceThatSymfonysRetryStrategyIsOn, the examplesreconnect.phpandcluster-failover.php
Development Setup
For contributors and development:
# Install dependencies composer install # Run static analysis and the default unit test suite after every modification composer test # Start NATS server for testing composer nats:start # Run unit tests with coverage composer test:unit # Set up functional tests composer test:functional:setup # Run functional tests composer test:functional # Stop NATS server composer nats:stop
Quick Start
1. Configure NATS Server
Ensure your NATS server has JetStream enabled:
nats-server -js
2. Set Up Transport in Symfony
Add the NATS transport to your Symfony Messenger configuration:
# config/packages/messenger.yaml framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost:4222/my-stream/my-topic' options: consumer: 'my-consumer' batching: 5 max_batch_timeout: 1.0 routing: 'App\Message\MyAsyncMessage': nats_transport
Tested by:
testReadmeDsnExamplesParseSuccessfully[README: quick-start transport],testReadmeConfigurationOptionsAreAccepted,testReadmeBatchingExamplesAreAccepted,testReadmeTimeoutExamplesAreAccepted
3. Configure Custom Serializers (Optional)
This transport ships with a high-performance IgbinarySerializer, but under the Symfony framework it is not selected automatically. Symfony Messenger always resolves a serializer and passes it to the transport factory - its framework default is the native PhpSerializer - so to use igbinary you must set the transport's serializer: key explicitly (shown below). The transport's own igbinary auto-selection (and the PhpSerializer fallback, with an E_USER_WARNING, when ext-igbinary is unavailable) only applies when you construct NatsTransport directly without passing a serializer, e.g. outside the framework.
Using IgbinarySerializer (recommended)
# config/packages/messenger.yaml framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost:4222/my-stream/my-topic' serializer: 'IDCT\NatsMessenger\Serializer\IgbinarySerializer' options: consumer: 'my-consumer'
The IDCT\NatsMessenger\Serializer\IgbinarySerializer service the serializer: key refers to is
registered by IDCTNatsMessengerBundle (see Installation), so the configuration
above works as-is. Only when the bundle is not enabled do you need to register it yourself, next to the
transport factory:
# config/services.yaml (only without the bundle) services: IDCT\NatsMessenger\Serializer\IgbinarySerializer: ~
Like Symfony's own serializers, it leaves out the stamps Symfony marks as not to be sent
(NonSendableStampInterface), such as the AckStamp a worker adds to the message it handles, so a message
Symfony re-sends for a retry or to the failure transport is serialized without them. Any serializer extending
AbstractEnveloperSerializer does the same.
Tested by:
createTransport_UsesProvidedSerializer,serialize_WithValidEnvelope_ReturnsSerializedString,decode_WithValidEncodedEnvelope_ReturnsEnvelope,testConstructorWithoutIgbinaryDoesNotCrash,theReadmesIgbinaryExampleGivesTheTransportTheIgbinarySerializer,theReadmesSetupWithoutTheBundleGivesAWorkingTransport,aSignedMessageSurvivesTheSerializerUnderEveryIdTheBundleGivesIt,encode_EnvelopeAWorkerIsHandling_EncodesWithoutItsNonSendableStamps,encode_WithNonSendableStamps_LeavesThemOutAndKeepsTheRest, Behat scenarioFailed message routed to Symfony failure transport via TERM with the igbinary serializer
Creating Custom Serializers
You can create your own serializer by extending AbstractEnveloperSerializer:
use IDCT\NatsMessenger\Serializer\AbstractEnveloperSerializer; use Symfony\Component\Messenger\Envelope; class MyCustomSerializer extends AbstractEnveloperSerializer { protected function serialize(Envelope $envelope): string { // Your custom serialization logic return serialize($envelope); } protected function deserialize(string $data): mixed { // Your custom deserialization logic return unserialize($data); } }
Tested by:
readmeCustomSerializerExample_EncodeDecode_RoundTrips,readmeCustomSerializerExample_DecodeInvalidBody_ThrowsException- the exact code above is compiled and exercised viaReadmeExampleSerializerin the unit tests.
For reference implementations, see:
src/Serializer/IgbinarySerializer.php- Binary serializationsrc/Serializer/AbstractEnveloperSerializer.php- Base class
4. Send Messages
use App\Message\MyAsyncMessage; use Symfony\Component\Messenger\MessageBusInterface; class MyController { public function __construct(private MessageBusInterface $bus) {} public function send(): void { $this->bus->dispatch(new MyAsyncMessage('Hello NATS!')); } }
Tested by:
testSendPublishesEncodedBodyWithoutHeaders,testSendUsesPublishWithHeadersWhenHeadersArePresent, Behat scenarioComplete message flow - send, check stats, consume, verify
5. Handle Messages
use App\Message\MyAsyncMessage; use Symfony\Component\Messenger\Attribute\AsMessageHandler; #[AsMessageHandler] class MyAsyncMessageHandler { public function __invoke(MyAsyncMessage $message): void { echo "Processing: " . $message->getText(); } }
Tested by: Behat scenarios
Complete message flow - send, check stats, consume, verify,Send and consume messages with a custom consumer name, andHigh-volume message processing with file output verification- handlers are exercised through realmessenger:consumeruns.
6. Consume Messages
symfony console messenger:consume nats_transport
Tested by: Behat scenarios
Complete message flow - send, check stats, consume, verify,Send and consume messages with a custom consumer name, andPartial message consumption with multiple consumers- the Behat context runsmessenger:consumeas a Symfony CLI process.
Examples
examples/ holds runnable scripts, one per behaviour, that use the transport directly against
the test server and print OK when what they show held:
| Example | Shows |
|---|---|
send-and-consume.php |
setup(), sending, the queue depth, receiving and acknowledging |
duplicate-protection.php |
DeduplicationIdStamp and deduplicate: true: a message sent again is stored once |
nats-retry.php |
retry_handler: nats with Symfony's default retry strategy: NATS redelivers a message whose handler failed, and the transport warns that the strategy is on |
retry-via-nak.php |
retry_via: nak: Symfony's retry strategy decides, and NATS redelivers the failed message after a NAK with Symfony's delay; a message Symfony gives up on is TERMed, with one copy in the failure transport |
retry-via-nak-edge-cases.php |
retry_via: nak at its edges: a delivery nobody settled, a message that cannot be decoded, the reject() of older Symfony Messenger releases, a message published with a retry count, and what the builder refuses |
retry-via-nak-past-max-deliver.php |
retry_via: nak with a low max_deliver: the retry of the last delivery it allows goes out as a copy, so Symfony's strategy runs to its end |
retry-via-nak-server-behaviour.php |
what the server does with a NAK retry it holds: a delay too long for it, a retry longer than stream_max_age, and a consumer with a backoff |
retry-via-nak-across-a-restart.php |
a NAK retry that waits while the server restarts is still redelivered, once |
retry-via-copy-or-nak.php |
the same failure under retry_via: copy and nak: a second consumer on the subject, max_ack_pending: 1, and a 0.3 s retry |
delayed-messages.php |
scheduled_messages: a delayed message arrives when its time has come, on NATS 2.12 and later; an older server is refused, at setup() and at the send |
request-timeout.php |
request_timeout, and sending again, with the same deduplication id, a message whose send timed out |
keepalive.php |
keepalive() from a SIGALRM handler, as messenger:consume --keepalive calls it |
connection-checks.php |
ping_after_idle, and a new dial once the connection is gone |
jwt-login.php |
jwt with nkey_seed: a user of a server in operator mode logs in; its JWT with another user's seed and an expired JWT are refused by the server, and the JWT alone when the transport is built |
nkey-login.php |
nkey_seed: a user the server knows by its public key logs in; a seed it does not know is refused by the server, and the public key alone or a seed next to a password when the transport is built |
token-login.php |
token: a client logs in with the server's token, and a wrong one is refused |
reconnect.php |
reconnect: true: a worker polls through a restart of its server, and auto_setup recreates the memory stream the restart lost |
cluster-failover.php |
reconnect: true on a cluster: a worker carries on through another server when the one it is connected to stops |
handler-failing-during-failover.php |
both retry modes on a cluster: a handler fails while the worker's server stops, and the worker carries on through another server; the message comes back at once as Symfony's retry copy, or nak_delay later in nats mode |
handler-succeeding-during-failover.php |
both retry modes on a cluster: a handler succeeds while the worker's server stops, and its ACK reaches the cluster through another server, so the message is not handled again |
stream-placement.php |
stream_placement_tags and stream_placement_cluster on a cluster, and staging a move with a replica change |
composer nats:start runs a three-server JetStream cluster for the last four next to the single test servers;
examples/README.md describes it, and
Using a NATS Cluster what they mean for an application.
composer nats:start
composer examples # or one at a time: php examples/keepalive.php
Tested by: CI runs
composer examplesafter the functional suite, on the latest and the oldest supported NATS.
Configuration Guide
DSN Format
nats-jetstream://[user:password@]host:port/stream-name/topic-name
Tested by:
testBuildWithValidDsnReturnsConfiguration,testBuildWithoutPathThrowsException,testBuildWithoutTopicThrowsException,createTransport_WithValidDsn_ReturnsNatsTransportInstance
Examples:
# Default port (4222) nats-jetstream://localhost/my-stream/my-topic # Custom port nats-jetstream://localhost:5000/my-stream/my-topic # With authentication nats-jetstream://user:password@localhost:4222/my-stream/my-topic # With query parameters nats-jetstream://localhost/my-stream/my-topic?consumer=worker&batching=10 # TLS transport scheme nats-jetstream+tls://localhost:4222/my-stream/my-topic
Tested by:
testReadmeDsnExamplesParseSuccessfully- each DSN above is parsed through the configuration builder via a dedicated data provider case.
Configuration Options
framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost:4222/my-stream/my-topic' options: # Consumer Configuration consumer: 'my-consumer' # Consumer group name (default: 'client') # Performance Tuning batching: 5 # Messages per batch (default: 1) max_batch_timeout: 1.0 # Timeout in seconds for batch fetching (default: 1) connection_timeout: 1.0 # Connection (dial) timeout in seconds (default: 1) ping_after_idle: 30 # Seconds the connection may go unused before the next # operation checks it with a PING and dials again if the # server does not answer (default: 30; 0 turns it off). # See "Losing the Connection". request_timeout: 10 # Seconds to wait for the server's reply to a publish, # an ack_sync ACK, setup() or a message count (default: 10) # Connection Resilience reconnect: false # Who recovers a lost connection (default: false) # false => the operation in flight fails and the transport dials # again on the next one; a worker exits and is restarted # true => the NATS client re-dials in the background and the # operations wait for it (see Automatic Reconnect) max_reconnect_attempts: null # Attempts per recovery before the client gives up # (null = the client's own default of 10). Positive integer. resend_on_reconnect: false # With reconnect: true, a send with a message id (deduplicate or # a DeduplicationIdStamp) whose server went away before it # answered goes out again on the recovered connection # (default: false; see Automatic Reconnect) # Consumer Flow Control & Lifecycle max_ack_pending: 1000 # Max delivered-but-unacked messages outstanding # (null = server default). Primary flow-control lever. inactive_threshold: 300 # Seconds of no pull activity before NATS removes the # durable consumer (null = server default). replay_policy: 'instant' # instant|original (default: null = server default 'instant') # β οΈ Immutable once the durable consumer exists: NATS rejects # the change, so the transport skips it and the option only # takes effect on a freshly created consumer. # Stream Retention Policies stream_max_age: 86400 # Max message age in seconds (0 = unlimited, default: 0) stream_max_bytes: 1073741824 # Max storage size in bytes (null = unlimited) stream_max_messages: 1000000 # Max number of messages in the stream (null = unlimited) stream_max_messages_per_subject: 1000 # Max number of messages retained per subject (null = unlimited) stream_max_message_size: 1048576 # Max size of a single message in bytes. # Must be a positive integer, at most 2147483647. # Unset means "leave it to the server": a new stream gets # JetStream's unlimited default, and an existing stream # keeps whatever limit it already has. To lift a limit on # an existing stream, change it in NATS directly. stream_max_consumers: 10 # Max consumers allowed on the stream (null = unlimited) # β οΈ NATS up to 2.11 refuses to change this on an existing # stream. When left unset the transport keeps whatever the # stream already has, so it never breaks an existing setup. # Stream Retention & De-duplication Behavior stream_retention: 'limits' # limits|interest|workqueue (default: null = server default 'limits'). # β οΈ Immutable once the stream exists: NATS rejects changing it, # so a changed value is ignored on an existing stream - recreate # the stream to change retention (see note below). stream_discard: 'old' # old|new - what to drop when a limit is hit (default: null = 'old') stream_duplicate_window: 120 # De-duplication window in seconds (null = server default). # Must be a positive integer; 0 is rejected because NATS # would substitute its own 2-minute default. # β οΈ NATS forbids a window larger than a finite max age, so # lowering stream_max_age below the stream's current window # rewrites the window down to match (the server does the # same at creation time). Raising stream_max_age again does # not restore it - set this option explicitly to do that. # Must not exceed stream_max_age when that is set. # Storage Backend stream_storage: 'file' # Storage type: 'file' or 'memory' (default: 'file') # β οΈ Immutable once the stream exists (like stream_retention). stream_compression: 'none' # none|s2 (default: null = server default 'none') stream_description: 'my stream' # Human-readable stream description (null = unset). # At most 4096 characters. # Stream Access Policy (null = leave the server default untouched) stream_deny_delete: false # Deny message deletion from the stream. # β οΈ One-way: NATS can turn this on but never off, so once # a stream denies deletes, setting false here is ignored. stream_deny_purge: false # Deny stream purge (one-way, exactly like deny_delete) stream_allow_direct: false # Allow direct get access stream_allow_rollup_headers: false # Allow Nats-Rollup headers # High Availability stream_replicas: 1 # Number of replicas of a new stream (default: 1). Unset, # an existing stream keeps its count. More than 1 needs a # JetStream cluster (see "Using a NATS Cluster"). stream_placement_cluster: null # Pin the stream to a named JetStream cluster # (null = leave the stream's placement untouched) stream_placement_tags: null # Only place the stream on servers carrying ALL of these # tags, e.g. ['ssd', 'eu-west'] (or 'ssd,eu-west' in a DSN) # (null = leave the stream's placement untouched) # Failure Handling Strategy retry_handler: 'symfony' # symfony|nats (default: symfony) # symfony => TERM on failed/rejected message # nats => NAK on failed/rejected message retry_via: null # copy|nak, only with retry_handler: symfony (default: null = copy) # copy => publish Symfony's retry copy as a new message # nak => NAK the failed delivery with Symfony's delay; NATS # redelivers it (see "Retrying through a NAK") # NATS-native Redelivery Tuning (mainly relevant with retry_handler: nats) nak_delay: 0 # Seconds to wait before NATS redelivers a NAK'd # message (default: 0 = immediate). Use to back off # instead of hot-looping a failing message. ack_wait: null # Seconds JetStream waits for an ACK before redelivering # (default: null = server default, ~30s). Raise it for # slow handlers to avoid premature redelivery. max_deliver: null # Max redelivery attempts before NATS gives up # (default: null = unlimited). Set it to stop a poison # message redelivering forever under retry_handler: nats. backoff: null # Redelivery schedule, in seconds, for a delivery nobody # settled, e.g. [1, 5, 30]. A plain NAK ignores it. Pairs # with max_deliver. (default: null) # Duplicate Protection deduplicate: false # Give every message an id JetStream deduplicates it by # within stream_duplicate_window (default: false) # Acknowledgement Mode ack_sync: false # Wait for server confirmation of each ACK (default: false) # false => fire-and-forget ACK (lower latency) # true => JetStream double-ack; a dropped ACK cannot # silently cause redelivery, at a latency cost # Scheduled / Delayed Messages (requires NATS >= 2.12) scheduled_messages: false # Enable scheduled message support (default: false) # When enabled, Symfony DelayStamp triggers NATS # scheduled message delivery via Nats-Schedule headers # Provisioning auto_setup: false # Provision the stream/consumer on first send/get (default: false). # When false (the default), run `messenger:setup-transports` # explicitly. When true, setup() runs once, lazily, on first use. # TLS Configuration tls_required: false # Force TLS for NATS connection (default: false) tls_handshake_first: false # Use TLS-first handshake mode (default: false) tls_ca_file: null # Path to CA certificate file tls_cert_file: null # Path to client certificate file tls_key_file: null # Path to client private key tls_key_passphrase: null # Passphrase for encrypted private key tls_peer_name: null # Override TLS peer name for certificate validation tls_verify_peer: true # Verify TLS peer certificate (default: true). Only # false, 0, no or off turns it off; any other value, # an empty one included, keeps it on # Additional Authentication token: null # NATS token authentication username: null # Overrides DSN username if provided password: null # Overrides DSN password if provided jwt: null # User JWT; needs nkey_seed, see NKey Authentication nkey: null # Leave unset: the public key is derived from nkey_seed nkey_seed: null # NKey seed (SU...) that signs the server's nonce; see NKey Authentication
Tested by:
testReadmeConfigurationOptionsAreAccepted(everything above but the redelivery tuning, TLS and authentication options, which the tests after it and the sections below cover),testReadmeBatchingExamplesAreAccepted,testReadmeTimeoutExamplesAreAccepted,testReadmeStreamRetentionExamplesAreAccepted,testBuildAcceptsAndNormalizesNewStreamAndConsumerOptions,testSetupPassesNewStreamPolicyOptions,testSetupPassesNewConsumerOptions,testBuildWithTlsAndAuthOptionsPropagatesToNatsOptions,testTlsVerifyPeerStaysOnUnlessExplicitlyDisabled,testTlsVerifyPeerInTheDsnStaysOnUnlessExplicitlyDisabled,testBuildAcceptsStreamPlacementOptions,testBuildWithReconnectOptionsPropagatesToNatsOptions
NKey Authentication
A NATS user can authenticate with an NKey instead of a password. The server keeps only the user's public key
(U...), and the client proves it holds the matching seed (SU...) by signing a nonce the server sends in its
INFO.
Give the transport the seed in nkey_seed, with no user, password or token in the DSN or the options:
framework: messenger: transports: orders: dsn: 'nats-jetstream://nats.example.com:4222/orders/orders.created' options: nkey_seed: '%env(NATS_NKEY_SEED)%' # a seed kept in a file (e.g. a mounted secret), its path in NATS_NKEY_SEED_FILE: # nkey_seed: '%env(trim:file:NATS_NKEY_SEED_FILE)%' # a seed that is optional, none for an unset or empty variable: # nkey_seed: '%env(default::NATS_NKEY_SEED)%'
On the server, the user is the public key of a pair you generate yourself. nsc generate nkey --user prints one, as does
nk -gen user -pubout from nkeys: the first line is the seed, the second the public key.
jetstream {}
authorization {
users = [ { nkey: U...your public key... } ]
}
- Only null, or leaving the option out, means no seed. Any other value that is not a user seed is refused when the
transport is built, without repeating it: an empty string, a list, a boolean, a mistyped seed, or the seed of an account,
operator, server, cluster or curve key. Read as no seed, an empty variable would connect with the other credentials
there are, as the server's
no_auth_user, or anonymously. - A username, password or token next to the seed, from the DSN or the options, is refused when the transport is built. A server with both kinds of users would log the connection in with the NKey and ignore them, even a wrong password, so the DSN could name one user while the connection is another. On a server that sends no nonce the client would refuse the login, as there is nothing to sign, though the password alone would have worked.
- For a user the server knows by its NKey, the seed is all the transport needs: the public key it sends is derived from
it, so leave the
nkeyoption unset. It may repeat the derived key, but a different one is refused when the transport is built, not on its first connect. So isnkeywithout a seed, which could never log in: the client needs a signer for the nonce, and the seed is where the transport gets one. - A seed needs a server that has NKey users or runs in operator mode, and in a cluster every server must list the user,
as the client can fail over to any of them. A server with NKey users refuses a seed it does not list with a bare
Authorization Violation. Any other server, a password, token or open one, sends no nonce, so the client refuses the login withNKey authentication requires server nonce from INFO, or the same words starting withJWTnext to ajwt. The reference CLI refuses it too. So set up the user on the server before you deploy the seed. Withreconnect: truethe client retries that failure, and the call takes about 43 seconds at the default attempts to reportReconnect attempts exhausted. - Signing needs the
sodiumextension, bundled with PHP since 7.2 and included in most builds. Without it the transport fails when it is built. The polyfillparagonie/sodium_compatdoes not replace it: the build then fails withFailed to derive Ed25519 key pair from NKey seed. - The transport's own parameters that can hold the seed are marked
#[\SensitiveParameter], so a failed build leaves it out of their stack frames. Symfony Messenger's factory marks its options only from 7.4.23 and 8.1.11, and the client marks nothing yet, so withzend.exception_ignore_args=0, PHP's default without a php.ini, frames of those two can still hold it. Keep that setting at 1 where stack traces are collected, asphp.ini-productiondoes. - The seed works in the DSN query too (
?nkey_seed=SU...), but prefer the option with an environment variable, as the recommendations say for every credential. A literal seed, in a DSN or in an option, ends up indebug:configand in a file of the compiled container, where a DSN or option that reads a variable does not. Keep the variable out of a committed.env, and do not log the full DSN. Astoken: nullhides?token=, an explicitnkey_seed: nullin the options hides a seed in the DSN query, so leave it out rather than copy it from the block above. A credential typed after a second?or a;in the query is refused when the transport is built, without repeating it, as it would be read as a part of the option before it: separate the options with&.
Tested by:
testBuildWithNkeySeedPassesASignerAndTheSeedsPublicKey,testBuildReadsTheNkeySeedFromTheDsnQuery,testWithoutTheSodiumExtensionASeedIsRefusedWhenTheTransportIsBuilt,testADisabledSodiumFunctionIsRefusedWhenTheTransportIsBuiltWithoutChainingTheError,testAFailureInsideSodiumIsRefusedWithoutSayingWhatASeedLooksLike,testBuildAcceptsAnNkeyThatIsTheSeedsPublicKey,testBuildRejectsAnNkeyThatIsNotTheSeedsPublicKey,testBuildRejectsAnNkeyWithoutASeed,testBuildRejectsASeedNextToAUsernamePasswordOrToken,testTheNkeyOptionIsNotDeprecated,testBuildRejectsAnInvalidNkeySeedWithoutRepeatingIt,testBuildSaysWhatASeedLooksLikeForAValueThatIsNotOne,testAMistypedSeedStaysOutOfTheFailure,testAContradictingNkeyLeavesTheSeedAndTheSignerOutOfTheFailure,testNoFrameOfAFailedBuildHoldsASecret,testNoFrameOfAFailedTransportCreationHoldsASecret,testEveryParameterThatCanHoldASecretIsMarkedSensitive,testASeedTypedIntoTheDsnPathStaysOutOfTheError,testTheConfigurationDoesNotKeepTheTextOfTheSeed,testBuildRejectsASeedThatIsNotAUserSeed,testBuildRejectsAnNkeySeedThatIsNotANonEmptyString,testBuildRejectsANkeySeedInTheDsnQueryThatIsNotANonEmptyString,testBuildWithANullNkeySeedHasNoSigner,testAnExplicitNullNkeySeedOptionHidesASeedInTheDsnQuery,theReadmesOptionalSeedFormMeansNoSeedForAnUnsetOrEmptyVariable,theReadmesOptionalSeedFormGivesASignerWhenTheVariableHasASeed,theReadmesRequiredSeedFormGivesASignerWhenTheVariableHasASeed,theReadmesSeedFileFormGivesASignerWhenTheFileHasASeed,theReadmesRequiredSeedFormIsRefusedForAnEmptyVariable,theReadmesSeedFileFormIsRefusedForAnEmptyFile,testASeedTypedIntoAnOptionsValueStaysOutOfTheError,testACredentialTypedAfterASecondQuestionMarkOrASemicolonIsRefusedWithoutRepeatingIt,testAValueThatOnlyLooksLikeACredentialInAQueryIsAccepted,testTheReadmeNamesNoUsableNkey,testCredentialsAreRefusedWhenBuiltOrReachTheServerInTheConnect,testASeedOnAServerThatSendsNoNonceCannotLogIn, the Behat scenarios ofnats_nkey.feature(real logins, in each form of the configuration above, also over TLS and mTLS and across a server restart, and the refusals through the console), the examplenkey-login.php. Not in the suite: a cluster failover with a seed, and the 43 seconds on a server that sends no nonce, which were measured by hand.
JWT Users (Operator Mode)
On a server in operator mode a user has a JWT as well as an NKey. Give the transport both: jwt is the user's JWT, and
nkey_seed is that user's seed. Both values are in the user's .creds file, under its BEGIN NATS USER JWT and
BEGIN USER NKEY SEED lines. The transport does not read the file itself, so pass the two values:
export NATS_USER_JWT=$(awk '/BEGIN NATS USER JWT/ { getline; print }' user.creds) export NATS_NKEY_SEED=$(awk '/BEGIN USER NKEY SEED/ { getline; print }' user.creds)
framework: messenger: transports: orders: dsn: 'nats-jetstream://nats.example.com:4222/orders/orders.created' options: jwt: '%env(NATS_USER_JWT)%' nkey_seed: '%env(NATS_NKEY_SEED)%'
jwtwithoutnkey_seedis refused when the transport is built: the client needs a signer for the server's nonce and cannot connect without one, and the seed is where the transport gets one. For a bearer JWT (nsc add user --bearer) the server does not check that signature, so there the seed is only what the client asks for, but it is required all the same.jwtmust be the JWT alone, which has three parts separated by dots. A seed or the whole text of a.credsfile there is refused when the transport is built, without repeating it, as the CONNECT would carry it to the server, in clear on a connection without TLS, and a server that traces logs it.- The seed must be the one of the JWT's user, unless the JWT is a bearer token. The server checks that when the transport
connects: a seed without its JWT, or another user's seed, gets a bare
Authorization Violation. - In operator mode JetStream is enabled per account, for example with
nsc edit account --name ACCOUNT --js-mem-storage 256M --js-disk-storage 1G, and the server needs the account's new JWT afterwards, for example from a newnsc generate config --mem-resolver. Without itmessenger:setup-transportsfails withJetStream not enabled for account, andmessenger:statsshows 0. That file has nojetstreamblock, and a server needs one as well, so addjetstream {}next to it, astests/nats/nats-jwt.confdoes: without it setup fails withNo responders for subject $JS.API.STREAM.CREATE.orders.tests/nats/auth/generate.shhas thensccommands for a whole test setup. - The transport keeps the JWT it was built with and never fetches a new one. At the JWT's expiry the server sends
User Authentication Expiredand closes the connection, which ends a running worker with that message. After that each new connection with the same JWT is refused withAuthorization Violation, which does not mention the expiry. A new JWT needs a new transport, so restart the workers with the new value.
Tested by:
testBuildPassesAJwtWithItsSeedToTheClient,theReadmesJwtFormGivesTheClientTheJwtAndASigner,testBuildRejectsAJwtWithoutASeed,testBuildRejectsAJwtOptionThatIsNotAJwtWithoutRepeatingIt,testBuildAcceptsAJwtWithSpaceAroundItAndBase64UrlCharacters,testCredentialsAreRefusedWhenBuiltOrReachTheServerInTheConnect, the Behat scenarios ofnats_jwt.feature(real logins on a server in operator mode, with a user's JWT and seed as in the configuration above and with a bearer JWT, across a server restart, and what is refused: a value that is not a JWT when the transport is built, and by the server another user's seed, a seed with no JWT, an expired JWT, an account without JetStream), the examplejwt-login.php. Not in the suite: a JWT that expires under a running worker, which was measured by hand on NATS 2.9.25 and 2.15.1.
Retry Handler Behavior
-
retry_handler: symfony(default) sendsTERMwhen a message fails during transport decoding or is rejected. Symfony's retry/failure transport then handles redelivery.Which failed messages the worker rejects depends on the Symfony Messenger release. Up to 7.4.21 and 8.1.9 it also rejects a message it sends for retry, so that message is TERMed next to its retry copy. From 7.4.22 and 8.1.10 it acknowledges that message once the retry copy is sent, so it is ACKed (symfony/symfony#67038). A
TERM, and theMSG_TERMINATEDadvisory JetStream publishes for it ($JS.EVENT.ADVISORY.CONSUMER.MSG_TERMINATED.<stream>.<consumer>), then marks only a failure Symfony treats as final (retries used up, an unrecoverable exception, no retries configured) or a message that could not be decoded. Counts or alerts built on those advisories therefore drop after such a Symfony upgrade, although the transport did not change. No message is lost: the retry copy is in the stream. -
retry_handler: natssendsNAKwhen a message fails during transport decoding or is rejected, so NATS redelivers the message itself.
In symfony mode, retry_via chooses how a retry Symfony decides on reaches NATS: as a copy (the default) or as a
NAK with Symfony's delay (see Retrying through a NAK).
With the defaults, Symfony's retry delays are dropped. Symfony's retry strategy waits between attempts
(1, 2 and 4 seconds with the framework's default strategy) by re-sending the failed message with a
DelayStamp. The transport applies a DelayStamp only when scheduled_messages is enabled, which needs
NATS 2.12 or later, and otherwise publishes the message at once. So with the defaults, each retry runs right
after the failure, all of them are used up within milliseconds, and nothing reports that the delays were
dropped. To keep the delays:
- on NATS 2.12 or later, enable
scheduled_messages(see Delayed / Scheduled Messages); - on any server, set
retry_via: nak: the transport NAKs the failed delivery with Symfony's delay, to the millisecond (see Retrying through a NAK); - or let NATS space out the attempts:
retry_handler: natswithnak_delay, bounded bymax_deliver, and withmax_retries: 0(see below).
NATS 2.9 to 2.11 have no message schedules, and the transport refuses scheduled_messages there (see
Delayed / Scheduled Messages): use retry_via: nak on those servers.
Tested by:
testSendWithDelayStampButScheduledMessagesDisabledPublishesNormally,testSendWithDelayStampPublishesToDelayedSubjectWithScheduleHeaders,testNakRetryNaksWithTheDelaySymfonyComputed,testSetupRefusesScheduledMessagesOnAServerWithoutSchedules
When NATS manages redelivery (retry_handler: nats), tune it with nak_delay, ack_wait, max_deliver, and backoff:
nak_delaydelays each NAK so a failing message backs off instead of redelivering immediately (a hot loop).max_delivercaps redeliveries. β οΈ Without it,retry_handler: natsredelivers a permanently-failing ("poison") message forever - setmax_deliverin production.backoffsets when NATS redelivers a delivery nobody settled (a crashed worker, anack_waitthat ran out), e.g.[1, 5, 30]seconds, and replacesack_waitwith its first step; pair it withmax_delivergreater than the number of backoff steps. On its own it does not space out failed messages: the transport NAKs those, and a plain NAK is redelivered at once. To a NAK with a delay (nak_delay), NATS addsbackoff[n] - backoff[0].ack_waitis how long JetStream waits for an ACK before considering a delivery failed and redelivering - raise it for handlers that legitimately take a while.
Symfony's retry strategy is ignored with retry_handler: nats. Symfony's retry listener runs for every
transport, and FrameworkBundle gives each one a retry strategy (max_retries: 3 by default). In nats mode the
transport does not publish the copy that retry sends, because NATS redelivers the original after the NAK, so
max_retries, delay and multiplier have no effect and max_deliver bounds the attempts. The worker then
settles the original: Symfony Messenger up to 7.4.21 and 8.1.9 rejects it, and 7.4.22, 8.1.10 and later
acknowledge it, since the retry was sent (symfony/symfony#67038).
The transport NAKs it either way. Set max_retries: 0 on the transport anyway: with retries left on, Symfony
still logs each failure as a retry it never makes ("Sending for retry #1") and never treats the message as
failed for good. To have Symfony's retry strategy decide while NATS redelivers, use retry_handler: symfony with
retry_via: nak instead. The transport warns about retries left on in nats mode through its logger (see
Logging), once per transport object, so once per worker, the first time it leaves out such a retry:
retry_handler=nats on stream "my-stream", subject "my-topic": Symfony's retry strategy retried a message that NATS redelivers itself. Set max_retries: 0 on this transport (Symfony still retries a RecoverableExceptionInterface), or have NATS carry out Symfony's retries with retry_handler: symfony and retry_via: nak (README, "Retry Handler Behavior").
The warning reads no configuration: a strategy that does not retry never sends a retry, so with max_retries: 0
it appears only when a handler throws a RecoverableExceptionInterface, which Symfony retries whatever
max_retries says. It works the same when retry_handler comes from a DSN in an environment variable.
β οΈ Up to 5.7.1 of this transport,
max_retries: 0is required withretry_handler: natson Symfony Messenger 7.4.22, 8.1.10 or later. Those releases acknowledge the message they send for retry, and 5.7.1 and earlier passed that ACK on to NATS: with retries left on, a message whose handler failed once was gone, neither redelivered nor in a failure transport (#90). Later releases NAK it.
A failure transport does not combine well with nats mode. Symfony sends a message there each time it treats a
failure as final: with max_retries: 0 that is every failed delivery, so a message that keeps failing lands
there up to max_deliver times, and with retries left on it never lands there. For a transport in nats mode,
rely on NATS's own handling of a message that keeps failing instead (max_deliver, and the JetStream
MAX_DELIVERIES advisory). With retry_via: nak, a failure transport
works as in symfony mode.
framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost:4222/my-stream/my-topic?retry_handler=nats&max_deliver=5&nak_delay=10' retry_strategy: max_retries: 0
Tested by:
testRejectUsesTermByDefault,testRejectUsesNakWhenRetryHandlerIsNats,testHandleFailedDeliveryUsesNakWithDelayWhenConfigured,testSetupAppliesConsumerRetryTuning,testBuildAcceptsNatsRetryTuningOptions,testBuildUsesRetryHandlerFromQuery,testNatsModeDoesNotPublishTheCopySymfonysRetrySends,testWorkerRetryInNatsModeNaksTheOriginalWithoutPublishingACopy,testWorkerInNatsModeNaksOnlyTheDeliveryItSentForRetry,testAckNaksADeliveryWhoseRetryCopyNatsModeLeftOut,testSymfonyModeAcksADeliveryItPublishedTheRetryCopyOf,testNatsModeWarnsOnceThatSymfonysRetryStrategyIsOn,testNatsModeWithoutSymfonysRetriesLogsNoWarning,testNoWarningWhenSendLeavesOutNoRetryCopy,testEachTransportWarnsOnceAboutItsOwnStreamAndSubject, Behat scenariosnats_nak.feature(withA message that fails once is redelivered with Symfony's default retry strategy),nats_term.featureandSymfony's retry strategy is ignored with the NATS retry handler, the examplenats-retry.php
Retrying through a NAK (retry_via: nak)
In symfony mode, Symfony's retry strategy decides whether and when a failed message is retried. By default
(retry_via: copy) its retry reaches NATS as a copy: a new message on the same subject, while the failed
delivery is settled for good, with a TERM or, from Symfony Messenger 7.4.22 and 8.1.10 on, an ACK. With
retry_via: nak the copy is not published: the transport NAKs the failed delivery with the delay Symfony
computed, and NATS redelivers the original.
framework: messenger: transports: async: dsn: 'nats-jetstream://localhost:4222/my-stream/my-topic?retry_via=nak' retry_strategy: { max_retries: 3, delay: 1000, multiplier: 2, max_delay: 60000 }
retry_via: copy (default) |
retry_via: nak |
retry_handler: nats |
|
|---|---|---|---|
| Decides whether to retry | Symfony's retry strategy | Symfony's retry strategy | NATS, up to max_deliver |
| Waits before a retry | Symfony's delay with scheduled_messages (NATS 2.12+), at a whole second rounded up; otherwise not at all |
Symfony's delay, to the millisecond, on any server | nak_delay |
| A retry is | a new message in the stream | the same message, redelivered | the same message, redelivered |
willRetry() and the retry count |
right | right, from the delivery count | false and 0, with max_retries: 0 |
| Given up on, or unrecoverable | a TERM, one failure-transport copy | a TERM, one failure-transport copy | a NAK, until max_deliver |
| Other consumers on the subject | handle each retry copy as well | see nothing extra | see nothing extra |
| While a retry waits | holds nothing | holds one of the consumer's max_ack_pending slots |
holds a slot for nak_delay |
max_deliver |
caps deliveries nobody settles | the same; past it a retry goes on as a copy | caps every redelivery |
Use nak on NATS 2.9 to 2.11, where a copy can't wait; for delays shorter than a second, or exact ones; when a
retry must not add a message to the stream; and when other consumers read the subject. Stay on copy when many
messages may wait at once with long delays: each waiting retry holds a max_ack_pending slot (1000 by default),
and once they are all held the consumer gets no new messages. Give the strategy a max_delay with nak, and keep
it below stream_max_age: a message that expires while it waits leaves the stream before its retry.
- The retry count is the count the message was published with plus its earlier deliveries, read from the
delivery's reply subject. A delivery nobody settled, after a crash or an
ack_waitthat ran out, counts as a retry. - The worker settles a retried message with
ack()(Symfony Messenger 7.4.22, 8.1.10 and later) orreject()(earlier releases). The transport NAKs it either way, after checking the connection, as for any failure. - A NAK cannot carry every retry. A retry copy without its delivery token is published as with
copy. So is the retry of the last deliverymax_deliverallows: NATS redelivers nothing after that delivery, so a NAK would end the message there, handled by nobody and in no failure transport. The copy goes on as a new message, with the retry count Symfony gave it and deliveries of its own, so Symfony's strategy runs to its end whatevermax_deliveris. That one retry does not wait for its delay unlessscheduled_messagesis on, it adds a message to the stream, and other consumers on the subject receive it.max_deliveris optional: leave it unset, or set it above the retries you expect, and it only caps deliveries nobody settles. - The builder refuses
retry_viawithretry_handler: nats, and withnaka positivenak_delay(the NAK carries Symfony's delay) andbackoff(NATS would addbackoff[n] - backoff[0]to each retry's delay). Runmessenger:setup-transportsafter taking abackoffout: a consumer that still has one keeps adding it. scheduled_messagesstill schedules aDelayStampthe application sets itself. Symfony's retries are not scheduled withnak, except the two retries above that go out as a copy.- A message this transport's igbinary serializer can't decode is TERMed at once. Symfony Messenger 8.1's own serializers return such a failure as a message, which goes through the retry strategy like any failure.
- A retry that waits while the server restarts is still redelivered once, up to a second early: the server keeps the time in whole seconds.
- On NATS 2.9 the server now and then reports a redelivery as a first delivery, when its NAK delay ends just as
a pull expires: a delay equal to
max_batch_timeout. Both are 1 second by default, and the retry strategy's jitter makes an exact match rarer. Symfony then repeats the first delay and allows one retry more. No message is lost, and NATS 2.10 and later do not do it.
docs/symfony_retry_via.md is the reference: the terms, the three setups delivery by delivery, what the server does with a NAK, and what was measured.
Tested by:
testRetryViaIsCopyUnlessSetToNak,testBuildWithInvalidRetryViaThrowsException,testBuildRefusesRetryViaInNatsMode,testBuildRefusesAPositiveNakDelayWithNakRetries,testBuildRefusesBackoffWithNakRetries,testRetryViaCopyStillAcceptsNakDelayAndBackoff,testNakRetryNaksWithTheDelaySymfonyComputed,testNakRetryNaksPlainlyForADelayOfZero,testNakRetryNaksWithTheDelayOfARecoverableException,testNakRetryCutsADelayThatWouldOverflow,testNakRetryFollowsSymfonysRetryStrategyThroughTheWorker,testNakRetryTermsADeliverySymfonyDoesNotRetry,testNakRetryTermsAnUnrecoverableFailure,testNakRetryTermsAnUndecodableMessage,testNakRetryPublishesTheCopyOfTheLastDeliveryMaxDeliverAllows,testNakRetryNaksADeliveryThatIsNotKnownToBeTheLast,testNakRetryRunsSymfonysStrategyToItsEndPastMaxDeliver,testGetGivesARedeliveredMessageItsEarlierDeliveriesAsRetryCount,testGetAddsTheEarlierDeliveriesToTheRetryCountTheMessageCarries, the Behat featurenats_retry_via.feature, and the examplesretry-via-nak.php,retry-via-nak-edge-cases.php,retry-via-nak-past-max-deliver.php,retry-via-nak-server-behaviour.php,retry-via-nak-across-a-restart.phpandretry-via-copy-or-nak.php. CI's lowest leg runs them on a Symfony Messenger that rejects a retried message (7.4.0 in the unit tests, 7.4.21 in the functional test app), the PHP 8.5 leg on one that acknowledges it.
Long-Running Handlers (--keepalive)
NATS redelivers a message that is not acknowledged within the consumer's ack_wait (30 seconds unless set),
possibly to another worker, even while a handler is still working on it. messenger:consume --keepalive
prevents that: every few seconds (5 by default, --keepalive=N to change it) Symfony has the transport tell
NATS that the message is still in progress, which restarts its ack_wait.
symfony console messenger:consume nats_transport --keepalive=10
Symfony sends the keepalive from a SIGALRM signal handler, which needs the pcntl extension and can
interrupt the handler at any point. The NATS client is asynchronous, and PHP does not let it wait inside a
signal handler, so the transport queues the in-progress acknowledgement there, and it goes out the next time
the event loop runs:
- A handler that waits on asynchronous work - any call to NATS through this library, such as dispatching a message to a NATS transport, or amphp-based I/O - lets it out at once, so the keepalive works as intended.
- A handler that only blocks - plain PHP work, or blocking calls such as PDO queries or curl requests -
lets it out only once it returns, too late to restart the timer. For such a handler, set
ack_waitlonger than its longest run instead.
Keep the keepalive interval well below ack_wait, so a keepalive that goes out a little late still arrives in
time.
Tested by:
testKeepaliveSendsInProgressForReplyToken,testKeepaliveFromASignalHandlerQueuesTheAcknowledgementInsteadOfWaiting,testKeepaliveDoesNotWaitForTheAcknowledgement,testKeepaliveWithoutAConnectionSendsNothingAndDoesNotDial,testKeepaliveWhoseAcknowledgementFailsRaisesNoUnhandledError, Behat scenarioA message handled for longer than ack_wait is not redelivered while keepalive runs
Important: Consumer Strategies
This is critical to understand before setting up multiple transport instances:
β οΈ Strategy A: Same Consumer, Batching = 1
Use when: Multiple instances should cooperate on the same consumer
# All instances use the same consumer with batching=1 transports: nats_worker_1: dsn: 'nats-jetstream://localhost/my-stream/my-topic' options: consumer: 'shared-consumer' # Same consumer name batching: 1 # MUST be 1 for shared consumers nats_worker_2: dsn: 'nats-jetstream://localhost/my-stream/my-topic' options: consumer: 'shared-consumer' # Same consumer name batching: 1 # MUST be 1 for shared consumers
Why batching must be 1:
- With explicit acknowledge (ACK) mode, only messages that are explicitly acknowledged are considered processed
- Multiple instances sharing the same consumer need to ACK individually
- Batching > 1 with multiple instances causes delivery conflicts
- Each instance should fetch and ACK one message at a time
Benefits:
- Automatic load balancing across instances
- NATS handles message distribution
- Guaranteed single processing per message
Tested by:
testReadmeBatchingExamplesAreAccepted(batching=1), Behat scenarioPartial message consumption with multiple consumers
β Strategy B: Different Consumers, Any Batching
Use when: Each instance needs independent message processing (duplicates allowed)
# Each instance uses a different consumer transports: nats_worker_1: dsn: 'nats-jetstream://localhost/my-stream/my-topic' options: consumer: 'worker-1-consumer' # Unique consumer per instance batching: 10 # Can use any batching nats_worker_2: dsn: 'nats-jetstream://localhost/my-stream/my-topic' options: consumer: 'worker-2-consumer' # Unique consumer per instance batching: 10 # Can use any batching
Why this works:
- Each consumer maintains its own state
- All messages are delivered to all consumers independently
- Each instance can use higher batching for better throughput
- Duplicate processing is expected (fan-out pattern)
Use cases:
- Event broadcasting to multiple systems
- Multiple independent processors
- Audit logging / event replay
Tested by:
testReadmeBatchingExamplesAreAccepted(batching=10), Behat scenarioPartial message consumption with multiple consumers
Batching & Timeouts
Batching Explained
- Higher batching: Better throughput, slightly higher latency
- Lower batching: Lower latency, slightly reduced throughput
- Optimal batching: Depends on message size and processing time
options: batching: 1 # Fetch 1 message at a time (low latency) batching: 5 # Fetch 5 messages (balanced) batching: 20 # Fetch 20 messages (high throughput)
Tested by:
testReadmeBatchingExamplesAreAccepted- values 1, 5, 10, 20, 50 are all verified.
Batch Timeout
Controls how long to wait for a batch to fill:
options: batching: 10 max_batch_timeout: 0.5 # Wait max 0.5s for batch to fill # Returns early if timeout reached
Tested by:
testReadmeTimeoutExamplesAreAccepted- values 0.5, 1.0, 2.0 are verified. Behat scenariosnats_batching.feature.
Example scenarios:
- If you set
batching: 10andmax_batch_timeout: 0.5 - If 10 messages arrive quickly, all are fetched immediately
- If only 3 messages arrive in 0.5s, return those 3
Connection Timeout
Controls the timeout for establishing the NATS connection: the first dial, and each new dial after the connection was lost (see Losing the Connection):
options: connection_timeout: 2.0 # Connection (dial) timeout in seconds
Tested by:
testReadmeTimeoutExamplesAreAccepted(1.0, 2.0, 3.0),testBuildWithConnectionTimeoutPropagatesMs
Purpose:
- Sets the timeout for the initial TCP/TLS dial and handshake when connecting to NATS
- Also bounds the PING that checks a connection before it is used again, after it sat idle or after an operation failed on it (
ping_after_idle), unlessrequest_timeoutis shorter: a server that does not answer within it is treated as gone - Does not bound how long an operation waits for the server's reply, which is
request_timeout; the batch fetch is bounded separately bymax_batch_timeout - Bounds each dial attempt: a dial that fails is tried three times, with pauses of 2 and 4 seconds in between
- Lower values fail faster only when an attempt hangs, as with an unreachable host (about 6 seconds plus three times the value in all); a refused connection fails after about 6 seconds whatever the value
- Higher values tolerate slower connection establishment
When to adjust:
- Increase for high-latency networks or geographically distant NATS servers
- Decrease for faster failure detection in local environments
- Default of 1 second works well for most local/regional deployments
Request Timeout
Controls how long an operation waits for the server's reply: the publish acknowledgement of send(), the
confirmation of an ACK with ack_sync, and the JetStream API calls of setup() and getMessageCount():
options: request_timeout: 30 # seconds (default: 10)
Tested by:
testRequestTimeoutDefaultsToTenSeconds,testBuildWithRequestTimeoutPropagatesMs,testBuildWithInvalidRequestTimeoutThrowsException,testReadmeTimeoutExamplesAreAccepted(10, 30)
A reply that does not arrive in time fails the operation with the client's TimeoutException. For send()
that does not prove the message was not stored: the server may have stored it and only its acknowledgement
was late, so dispatching the message again can store it twice, unless it carries a deduplication id (see
Duplicate Protection). Raise the value for a slow or distant server,
or for a stream whose writes can take long, such as a replicated one under load; lower it to fail faster.
The wait of a pull in get() is bounded by max_batch_timeout instead (the client gives up a second later
on a pull the server does not answer at all), and the dial by connection_timeout.
Duplicate Protection
A send() that timed out may still have stored the message (see Request Timeout), so
dispatching it again can store it twice. JetStream drops a message whose Nats-Msg-Id it already stored within
the stream's duplicate window (stream_duplicate_window, 2 minutes unless set), and the transport fills that
header from a DeduplicationIdStamp:
use IDCT\NatsMessenger\Stamp\DeduplicationIdStamp; // Dispatched again with the same id after a send() that failed, the message is stored once. $bus->dispatch(new OrderPlaced($orderId), [new DeduplicationIdStamp('order-placed-' . $orderId)]);
With deduplicate: true the transport gives every message an id of its own on its first send. The id is sent
with the message, so a copy sent again keeps it, which covers what Symfony sends again itself: a worker that
stopped after a retry or failure-transport send that timed out, the server having stored the copy, gets the
original redelivered by NATS after ack_wait and sends the same copy again, which JetStream drops. For that,
stream_duplicate_window has to be longer than ack_wait plus the handler's run. Each of Symfony's retries,
and the copy for a failure transport, gets an id of its own, so none of them is taken for the original. An id
you add yourself, as above, is used with the option off as well, and covers a message your application
dispatches again as a new envelope, which would otherwise get a new id.
The header holds the id together with the transport's subject and the retry count, and on a failure-transport copy the transport the message failed on. JetStream deduplicates per stream whatever the subject, and Symfony hands each transport a message is routed to the envelope the one before returned, id included, so transports that share a stream (see Multi-Subject Streams) each keep their copy.
options: deduplicate: true # default: false
With reconnect: true, resend_on_reconnect: true puts the id to one more use: a send whose
server went away after it took the message and before it answered goes out once more on the recovered connection,
where JetStream drops it if the first copy was stored.
Tested by:
testSendWithDeduplicationStampsTheEnvelopeAndPublishesItsMessageId,testSendKeepsTheDeduplicationIdTheEnvelopeCarries,testSendGivesEachRetryAndTheFailureTransportCopyAMessageIdOfItsOwn,testSendWithoutDeduplicationSendsNoMessageIdAndAddsNoStamp,testSendUsesAnApplicationDeduplicationIdWithTheOptionOff,testSendDelayedMessageWithDeduplicationPublishesItsMessageId,testDeduplicateIsOffByDefaultAndCanBeEnabled,testSendGivesTheCopiesForTwoTransportsOnOneStreamMessageIdsOfTheirOwn,testSendGivesTheFailureCopiesOfTwoTransportsMessageIdsOfTheirOwn, Behat scenariosA message dispatched again with the same deduplication id is stored once,Distinct messages are all stored with the deduplicate optionandA message routed to two deduplicating transports on one stream is stored for each
Losing the Connection
Unless reconnect is on, the transport does not reconnect in the background, and
nothing in Symfony calls close(). When the connection has been lost, the next operation dials again
instead:
- A connection is lost when the NATS server restarts, when the network drops it, or when the server closes it because the client stopped answering its pings. A PHP process answers them only while it is inside a transport call, so a server with default settings drops a connection that has been idle for about 6 to 8 minutes; a load balancer or NAT gateway may drop an idle connection sooner.
- The operation that runs into the lost connection fails. For
send()that means the outcome is unknown: the server may or may not have stored the message. Once the client reports the connection closed, the next operation opens a new one, on a cluster possibly to another server (see Using a NATS Cluster). A consumer worker exits on the failedget()and its supervisor starts a new one; a process that sends for longer than one request (a web app in worker mode such as FrankenPHP or RoadRunner, a daemon, a handler that dispatches) keeps working from the next operation on. - With
auto_setup, the new connection is verified in the same call that dials it, and a dial that fails surfaces as the connection error itself.
A connection that was dropped while it sat idle would make the next operation the one that fails. To avoid
that, an operation that finds the connection unused for longer than ping_after_idle (30 seconds by
default) first checks it with one PING. If the server does not answer within connection_timeout (or
request_timeout, when that is shorter), the transport closes that connection and runs the operation on a
new one, so the first message after a quiet period goes out instead of failing. The PING costs one round
trip, and only after a quiet period. A lower value also catches a server restart between two operations
further apart than it, at the cost of that round trip more often. The same check follows an operation that
failed on the connection - a JetStream error reply does not count, since the server answered it - because
the client can keep reporting a dead connection open, and a send to a connection that silently stopped
delivering only times out. A pull from such a connection ends empty, which reads as no messages rather than
a failure, but the server never answered it: a server that is there ends every pull itself, after
max_batch_timeout at the latest. So the check follows such a pull too, and the pull after it runs on a new
connection. The NAK or TERM of a message whose handler failed runs the check too, even right after a pull: the
handler may have run into the server going away, and the client reads the socket only while an operation waits
for a reply, so a server that closed the connection meanwhile leaves that close unread, and an acknowledgement
written then was taken by the socket and lost (see
A Handler Failing During the Loss). Since client 2.28.0, which 5.8.0
requires, the client notices such a close itself before it writes, but the check still matters: with
reconnect: false the client would fail the NAK with Reconnect is disabled and end the worker, where the check
dials another server first, and a close behind data not read yet, such as a TLS server's close_notify, is one the
client cannot notice that way. That costs one round trip per failed message; the ACK of a handled message is not
checked. ping_after_idle: 0 turns all of these checks off.
keepalive() never checks: Symfony calls it from a signal handler.
options: ping_after_idle: 30 # seconds; 0 turns the check off
Tested by:
testOperationAfterTheClientClosedDialsAgain,testAutoSetupVerifiesTheNewConnectionBeforeTheSameCallPublishes,testFailedDialAfterTheClientClosedSurfacesAsTheConnectionErrorAndIsRetried,testIdleConnectionIsCheckedWithAPingAndKeptWhenTheServerAnswers,testIdleConnectionThatDoesNotAnswerThePingIsReplacedBeforeTheOperation,testPingUnansweredWithinTheConnectionTimeoutReplacesTheConnection,testRecentlyUsedConnectionIsNotPinged,testPingAfterIdleZeroTurnsTheCheckOff,testKeepaliveNeitherPingsNorDials,testOperationThatFailedOnTheConnectionMakesTheNextOneCheckItFirst,testJetStreamReplyDoesNotMakeTheNextOperationCheckTheConnection,testPullTheServerDidNotAnswerMakesTheNextOperationCheckTheConnection,testOnlyAPullTheServerDidNotAnswerMakesTheNextOneCheckTheConnection,testIdleTimeCountsFromTheEndOfAnEmptyPull,testSettlingAFailedDeliveryChecksTheConnectionFirst,testPingAfterIdleZeroTurnsTheCheckBeforeSettlingAFailedDeliveryOff,testBuildWithInvalidPingAfterIdleThrowsException, Behat scenariosA message sent after the server dropped the idle connection goes outandWithout the check the message sent after the server dropped the connection fails
Subscription Limits
The transport holds up to two subscriptions on its connection: the reply inbox of its requests, which the first
send(), ack_sync acknowledgement, setup() or getMessageCount() on a connection subscribes, and the reply inbox
of the pull in progress. NATS caps the subscriptions a connection may hold at the smallest of the server's
max_subscriptions, the account's limit and the user's. With a cap of one, the server refuses the pull's inbox with
maximum subscriptions exceeded, and every pull after the first request on that connection fails. get() then
throws Symfony's TransportException, with the client's SubscriptionLimitException as its previous exception:
Cannot pull from NATS stream "orders", consumer "worker": the subscription limit was exceeded. This transport requires capacity for at least two subscriptions per connection. Check server, account and user limits.
Like any exception from get(), it ends a worker's run. Raise the limit to at least two. The operation after the
failure checks the connection first, as after any other failure on it (see
Losing the Connection): the server sends the same error when an account's limit is lowered
below what the connection holds, and closes the connection right behind it.
Tested by:
testAtALimitOfOneThePullFailsWithATransportExceptionNamingTheLimit,testAtALimitOfOneAPullWhoseRefusalTheHeartbeatReadsFailsTheSameWay,testAtALimitOfTwoPullsDeliverAndAcknowledge,testALimitErrorTheServerClosesTheConnectionBehindKeepsTheCheckBeforeTheNextOperation,testALimitErrorTheServerClosesTheConnectionBehindLeavesAClosedClientThatIsReplacedWithoutACheck,testPullRefusedAtTheSubscriptionLimitFailsWithATransportExceptionAndTheNextOperationChecksTheConnection,testAutoSetupRetriedPullRefusedAtTheSubscriptionLimitFailsWithATransportException,testPullThatFailedOnTheConnectionOtherwisePropagatesAsItIs
Automatic Reconnect
Who recovers a lost connection is a choice between two modes.
reconnect: false (the default): Symfony owns recovery. The NATS client runs with reconnect off. The
operation that runs into the lost connection fails at once, the next one dials again, a consumer worker exits
on the failed get() and its supervisor starts a new one, and Symfony's retries and failure transport take
care of the message. See Losing the Connection; nothing below applies.
reconnect: true: the client owns recovery. The NATS client re-dials in the background as soon as it
notices the loss, with exponential backoff (100 ms doubling up to 10 s, with jitter) and up to
max_reconnect_attempts times per recovery (the client's default is 10), replays its subscriptions and flushes
the frames it buffered meanwhile. The transport steps back from recovering the connection itself:
options: reconnect: true # the client recovers a lost connection (default: false) max_reconnect_attempts: 20 # attempts per recovery (default: null = the client's default of 10)
In a DSN: ?reconnect=true&max_reconnect_attempts=20.
- A client in the middle of its recovery is left alone: no PING, no dial of the transport's own. The
operation waits for the recovery within its own budget: a request (
send(), anack_syncACK,setup(),getMessageCount()) withinrequest_timeout, a pull withinmax_batch_timeoutplus one second, after which it ends empty. The budget is per request, not per method:setup()makes three or more requests, a publish retries a missing responder three times on its own, and the first dial of a process is not bounded by it at all. - The PING check of Losing the Connection still runs, and when the PING fails, what
the client made of the connection decides. A client that is recovering, or that already recovered (its
reconnect count moved, so the PONG belonged to the lost connection), keeps its connection. A client that
still reports the same connection open is replaced, as without the option: nothing in the client recovers
a connection it does not know to be lost before its heartbeat has gone unanswered three times, about 90 s at
the client's defaults, and the heartbeat only runs while an operation holds the event loop. A client that
gave up is Closed and dialled again. A PHP process runs the event loop only inside a transport call, so the
client notices a lost connection when an operation writes to it or when this check runs: a
ping_after_idleshorter than the quiet periods between operations makes the first operation after one check first and go out on the recovered connection, where without the check it writes into the dead socket, waitsrequest_timeoutand fails, and only the next operation goes out. A connection the server closed is the exception since client 2.28.0: the client sees that close before it writes, and the operation goes out on the recovered connection without the check; a connection that went silent, or a close behind data not read yet (a TLS server'sclose_notify), it still cannot see. - With
auto_setup, provisioning is verified again once the client reconnected, because the stream may have gone with the server (a memory stream over a restart, a failover to a server without it). A send begun while the client was recovering can still land on a server without the stream; when its publish fails with the client's 503 "No JetStream responder",setup()runs once and the same frame is published once more, then a second failure propagates. Nothing answered the first publish, so the second cannot duplicate it. - When the recovery gives up, the client is Closed and the acknowledgements it buffered during the outage are
discarded; the messages come back after
ack_wait, counting one delivery againstmax_deliver. The client reports the discard at error level through the logger the bundle gives it (Reconnect exhausted: N bytes of buffered publishes were discarded, under Monolog'smessengerchannel). The next operation dials again, and with the option on that dial runs the client's attempt loop inside the operation. Exhaustion propagates as the client'sConnectionException("Reconnect attempts exhausted") rather than being flattened into an empty queue, so a worker ends and its supervisor restarts it. The same loop runs for a failed first dial, somessenger:setup-transportsagainst a server that is down fails only after it. - What a worker gets: it polls through an outage shorter than one recovery cycle without exiting, every pull
ending empty after
max_batch_timeoutplus a second. That includes the last moments of a graceful stop, which on NATS 2.15 leave the connection open while the server stops each durable consumer on a file stream, in about 10 ms each, and lame duck mode (nats-server --signal ldm, SIGUSR2), in which the server stops JetStream at once and keeps the connections for its grace period (lame_duck_grace_period, 10 s by default) before it closes them and exits: a pull written then gets 503, the same status a missing consumer gives. After a 503 the client asks for the consumer once. The server going away leaves that question unanswered, and closes the connection or announces lame duck mode, and the pull is an empty batch; a server that stays answers it, and the 503 keeps its reading as a missing consumer (see Note on a server that stops). Up to 5.8.0 that pull ended the worker, withJetStream pull request ended with status 503or, withauto_setup,Failed to setup NATS stream(#106). Under a supervisor the worker starts again.max_reconnect_attemptssizes the cycle in attempts, not in seconds: the client's default of 10 gives about 109 s against a server that refuses connections, where a refused dial costs about 6 s. Anack_syncacknowledgement that runs into the outage still fails afterrequest_timeout, which ends the worker's run as any exception fromack()does: synchronous acknowledgement keeps its meaning. The retry copy Symfony sends when a handler fails in the defaultretry_handler: symfonyis a publish: one sent after the server closed the connection goes out on the recovered connection (client 2.28.0), but one the server took and could not answer before it went away fails afterrequest_timeoutand ends the run, unlessresend_on_reconnectsends it again (below; see also A Handler Failing During the Loss). - An ACK, NAK or TERM written during a recovery is buffered by the client, and
ack()orreject()returns at once. That holds too when its own write is the first to notice the lost connection while the client still reports it open: the write starts the recovery, and the frame goes out on the connection the recovery makes. Before client 2.27.1, which the transport requires since 5.7.1, such a write ran the recovery inline, so oneack()orreject()could wait for a whole attempt loop (client issue #244). Since client 2.28.0, which 5.8.0 requires, the same holds for an acknowledgement written after the server closed the connection while nothing read it, which the socket used to take and lose.keepalive()only queues its frame, as it must inside a signal handler, and never waits, PINGs or dials. - A server that stops gracefully ends every pull in flight with
409 Server Shutdown, and a consumer whose leader moved ends them with409 Leadership Change. Both are empty batches, in either mode, and no longer read as a missing consumer; after a shutdown the next operation checks the connection first.
Measured on nats-server 2.14.2 with client 2.27.0 and PHP 8.5, request_timeout 10, connection_timeout 1,
max_batch_timeout 1, auto_setup with a memory stream, the server stopped with docker stop for about 53 s
and the same transport object throughout. A refused dial on localhost costs about 6 s:
| Operation while the server is down | max_reconnect_attempts 10 (the client's default) |
max_reconnect_attempts 2 |
|---|---|---|
send() in flight at the stop |
10.0 s, TimeoutException |
10.0 s, TimeoutException |
next send() |
10.0 s, TimeoutException (the client is still recovering) |
2.4 s, Reconnect attempts exhausted (the recovery gave up inside the wait) |
get() |
2.0 s, empty | 18.4 s, Reconnect attempts exhausted |
plain ack() |
0.00 s, buffered | 18.4 s, Reconnect attempts exhausted |
getMessageCount() |
20.0 s (two requests), 0 | 18.4 s, 0 |
send() after those |
10.0 s, TimeoutException |
18.4 s, Reconnect attempts exhausted |
first send() once the server is back |
2.8 s, stored; the memory stream recreated | 0.13 s, stored; the memory stream recreated |
With 10 attempts the recovery cycle outlasted the outage, so every operation waited within its own budget and none dialled; once the server was back, the first send waited for the client's next attempt, at the 10 s cap of the backoff by then. With 2 attempts the client gave up about 12 s into the outage, and from then on every operation dialled inline, one dial and two attempts, and failed with the client's exception. Before this mode existed in the transport, the option as first proposed ran the whole attempt loop inside every operation during an outage, 109 s each at the defaults. The figures are measurements, not bounds: the option bounds attempts per recovery, not time, and a process can run any number of recovery cycles over its life.
When to enable: long-running workers where a restart is expensive, against a NATS cluster whose nodes
restart or fail over (see Losing a Server). Not for web requests that dispatch: a
request handler stalls for a whole attempt loop when it runs into an outage once the client gave up. Leave
it off to rely on the process supervisor and Symfony's retries, and to have messenger:setup-transports
fail fast when NATS is unreachable.
A send in flight when its server goes away (resend_on_reconnect). A send() whose server took the message
and went away before it answered, as a stream's leader that stops, waits out request_timeout on the recovered
connection and fails with TimeoutException, its outcome unknown: the server that took it can no longer answer.
With resend_on_reconnect: true the client writes the same publish once more on the
recovered connection, within the same request_timeout, when it carries a Nats-Msg-Id, which deduplicate: true
or a DeduplicationIdStamp gives it (see Duplicate Protection). JetStream drops it when
the first copy was stored and answers it either way, so the send succeeds; Symfony's retry copy is covered the same
way. Only a replaced connection triggers it, never a slow answer, at most once per publish, and a publish without
an id is never sent again (client 2.28.0,
client issue #249).
options: reconnect: true # required with resend_on_reconnect deduplicate: true # or a DeduplicationIdStamp of your own on each message resend_on_reconnect: true # default: false
In a DSN: ?reconnect=true&deduplicate=true&resend_on_reconnect=true. It is off by default because of two limits,
the client's: the stream's duplicate window has to be longer than request_timeout (the 2-minute default is; the
transport refuses a stream_duplicate_window that is not, but cannot check the window of a stream it does not
configure), and on a NATS 2.10 cluster a new stream leader can take the publish before it has applied what the old
leader replicated, so a copy may be stored twice there.
Tested by:
testBuildTurnsClientReconnectAndPedanticModeOff,testBuildWithReconnectOptionsPropagatesToNatsOptions,testBuildWithReconnectFromDsnQueryString,testBuildKeepsTheClientReconnectAttemptDefaultWhenOnlyReconnectIsEnabled,testBuildWithInvalidMaxReconnectAttemptsThrowsException,testResendOnReconnectIsOffByDefault,testBuildWithResendOnReconnectPropagatesToNatsOptions,testBuildWithResendOnReconnectWithoutReconnectThrowsException,testBuildWithResendOnReconnectAndADuplicateWindowNotLongerThanTheRequestTimeoutThrowsException,testBuildWithResendOnReconnectAcceptsALongerOrAnUnsetDuplicateWindow,testASendWhoseServerWentDownBeforeItAnsweredIsWrittenAgainOnlyWithResendOnReconnectAndAMessageId,testBuildPassesTheLoggerToTheClientAndNoneByDefault,testReconnectModeLeavesAClientInTheMiddleOfItsRecoveryAlone,testReconnectModeReadsAFailedPingAgainstTheClientsRecovery,testReconnectModeTakesARecoveredConnectionAsChecked,testWithoutReconnectAFailedPingReplacesTheConnectionAndTheReconnectCountIsNeverRead,testReconnectModeVerifiesProvisioningAgainAfterTheClientReconnected,testReconnectModeReprovisionsOnceWhenTheFirstPublishAfterTheReconnectFindsNoResponder,testReconnectModeDoesNotRetryAPublishOutsideTheNoResponderCase,testAPullTheServerEndedForItsShutdownIsEmptyAndTheNextOperationChecksTheConnection,testAPullEndedByALeadershipChangeIsEmptyAndTheNextOneDoesNotCheckTheConnection,testA409ReportingAConfigurationErrorStaysVisible,theBundleHandsTheApplicationsLoggerToTheFactory,testAnAcknowledgementWhoseWriteFindsTheConnectionLostReturnsAtOnceAndTheRecoverySendsIt, the real-client tests intests/unit/ReconnectModeTest.php, and the Behat scenariosA worker keeps consuming across a server restartandA message sent after a server restart goes out on the reconnected connection and the memory stream is recreated; the examplesreconnect.phpandcluster-failover.php(a failover in a three-server cluster)
Stream Configuration
Retention Policies
Control how long messages are kept in the stream:
options: # By age (24 hours) stream_max_age: 86400 # By total size (1GB) stream_max_bytes: 1073741824 # By total message count across the entire stream (NATS: max_msgs) stream_max_messages: 1000000 # By message count per individual subject (NATS: max_msgs_per_subject) stream_max_messages_per_subject: 1000 # Unlimited (default) stream_max_age: 0 stream_max_bytes: null stream_max_messages: null stream_max_messages_per_subject: null
Tested by:
testReadmeStreamRetentionExamplesAreAccepted- all retention options above are verified. Behat scenariosnats_stream_limits.feature.
Note:
stream_max_messageslimits the total number of messages stored in the stream (maps to NATSmax_msgs), whilestream_max_messages_per_subjectlimits messages retained per individual subject (maps to NATSmax_msgs_per_subject). The per-subject limit is especially useful with multi-subject streams to prevent one high-volume subject from dominating retention.
High Availability
options: # Single replica (no redundancy) stream_replicas: 1 # 3 replicas (recommended for production) stream_replicas: 3
More than one replica needs a JetStream cluster: a standalone server refuses to create such a stream. Left
unset, stream_replicas gives a new stream one replica and leaves the count of an existing stream alone, so
a stream scaled up in NATS is not scaled back down by the next setup(). See
Using a NATS Cluster.
When setup() changes the replica count of an existing stream, NATS moves the stream's durable consumer to the
stream's new set of servers after it has answered the update. A consumer whose leader was on a server the stream
left has no leader for a moment, and the server answers a request for it with err_code 10008, "JetStream system
temporarily unavailable" (measured on nats-server 2.10 going from 2 replicas to 1). In the same moment a request
can also be told that the stream is not found (err_code 10059), or get no answer at all. setup() asks for the
consumer again on either code, every 100 ms for up to 5 seconds, and once more after a request that got no
answer within request_timeout, so messenger:setup-transports does not fail on that moment. A cluster that
stays unavailable longer, a stream that really is gone, and a second request without an answer still fail the
setup with the server's message.
Tested by:
testReadmeStreamRetentionExamplesAreAccepted(replicas 1 and 3),testSetupPassesConfiguredStreamOptions,testSetupPreservesExistingReplicaCountWhenStreamReplicasNotConfigured,testSetupOverridesExistingReplicaCountWhenStreamReplicasExplicitlyConfigured,testSetupAsksForTheConsumerAgainWhileTheClusterIsTemporarilyUnavailable,testSetupCreatesTheConsumerAgainWhileTheClusterIsTemporarilyUnavailable,testSetupWaitsBeforeAskingTheTemporarilyUnavailableClusterAgain,testSetupGivesUpOnATemporarilyUnavailableClusterAfterFiveSeconds,testSetupDoesNotAskForTheConsumerAgainAfterAnotherJetStreamError,testSetupAsksForTheConsumerAgainWhileTheMovedStreamIsReportedAsNotFound,testSetupCreatesTheConsumerAgainWhileTheMovedStreamIsReportedAsNotFound,testSetupGivesUpOnAStreamStillReportedAsNotFoundAfterFiveSeconds,testSetupReadsAMissingConsumerAsNoConsumerWithoutAskingAgain,testSetupAsksForTheConsumerOnceMoreAfterARequestTheClusterDidNotAnswer,testSetupCreatesTheConsumerOnceMoreAfterARequestTheClusterDidNotAnswer,testSetupGivesUpOnASecondConsumerRequestTheClusterDidNotAnswer,testEachConsumerRequestGetsItsOwnSecondTryAfterAnUnansweredOne,testSetupGivesUpOnATemporarilyUnavailableClusterAfterAnUnansweredRequestPastFiveSeconds, the examplestream-placement.php(from 2 replicas to 1 on the test cluster)
Stream Placement
In a JetStream cluster you can pin a stream to a named cluster and/or to the servers that carry specific
tags (server_tags in the NATS server configuration). A stream is only placed on servers carrying
all of the listed tags:
options: # Only on servers tagged both "ssd" and "eu-west" stream_placement_tags: ['ssd', 'eu-west'] # In the "east" cluster of a super-cluster stream_placement_cluster: 'east' # Both at once stream_placement_cluster: 'east' stream_placement_tags: ['ssd']
In a DSN the tags are a comma-separated list: ?stream_placement_tags=ssd,eu-west.
Tested by:
testBuildAcceptsStreamPlacementOptions,testBuildSplitsCommaSeparatedStreamPlacementTags,testBuildParsesStreamPlacementFromDsnQueryString,testSetupPassesStreamPlacementOnCreate,testSetupUpdateAppliesConfiguredStreamPlacementToAnExistingStream,testSetupUpdatePreservesServerStreamPlacementWhenNotConfigured,testSetupUpdateMergesConfiguredPlacementIntoTheLivePlacement,testSetupExplainsHowToStageAMoveAndScaleRejection,testAutoSetupRepeatsARejectedMoveAndScaleUpdateOnEverySend, Behat scenarios innats_stream_placement.feature, and the examplestream-placement.phpon a three-server cluster
- Both options default to
null, which leaves the stream's placement untouched: a new stream gets the server's default placement and an existing stream keeps whatever placement it already has. This matters because a JetStream update that omitsplacementclears it, so the transport echoes the live value back whenever the options are unset. - On an existing stream each option replaces its own field of the live placement and nothing else:
stream_placement_tagsreplaces the tag list and keeps a cluster pin set in NATS, andstream_placement_clusterreplaces the cluster and keeps the live tags. Neither option can clear the other field, so to remove a placement, or one half of it, change it in NATS directly. - nats-server compares the stored placement with the new one field by field and treats any difference as
a request to place the replicas again, even when the servers they are on already satisfy the new
constraints, and even when only the order of the tags changed. The transport therefore keeps the live
tag list as the server stores it whenever the configured tags name the same set, and compares the tags
exactly:
SSDandssdmatch the same servers, but the server'sunique_tagbypass (below) is case-sensitive, so a tag that differs only in case is sent as a change. The server accepts a new placement at once andsetup()returns; the replica migration runs in the background (nats stream infoshows the replicas as not current until it finishes). - In a super-cluster a placement that names tags but no cluster is resolved in the cluster the client
running
setup()is connected to, then in the clusters that one lists as alternates, on create and on a move alike. A tags-only change can therefore relocate the stream to another cluster. Setstream_placement_clusterto keep the stream where it is: the transport does not fill the cluster in from the live stream, because a pin it made up would show innats stream infoas configuration nobody asked for, and the options could never clear it again. - NATS refuses to change the placement and
stream_replicasin the same update (err_code 10123), andsetup()fails with the server's message and the staging steps. Stage the two changes: runmessenger:setup-transportswith only one of them (for example the placement options as they were and the newstream_replicas), wait untilnats stream infoshows every replica as current, then run it again with the other change. Running the same combined configuration twice is not staging. Withauto_setup: truethe transport runssetup()before the firstsend()orget()of a process, so every one of them repeats the rejected update and fails until the configuration is staged; nothing is published or consumed meanwhile. The transport does not split the update itself, because asetup()that applied one half and returned would count as done and the other half would never be applied. - Exclusion tags and the
unique_tagbypass (NATS 2.12+). A tag prefixed with!excludes the servers that carry it, for example!disk:hdd. The server strips the!before it looks at the cluster'sunique_tagsetting, and any placement tag that starts with that configured prefix switches the uniqueness constraint off for the stream (nats-server's own comment: "disable uniqueness check if explicitly listed in tags"). So withunique_tag: "zone:", the tag!zone:ignoreexcludes nobody and only lifts the requirement that replicas sit in distinct zones, which lets a 3-replica stream be created on a cluster with fewer than 3 zones. A positive tag that every server carries and that starts with the prefix, such aszone:anyadded to each server'sserver_tags, has the same effect on any server version. On 2.11 and older the!is not stripped, so!zone:ignoreis a mandatory tag nothing matches and creation fails withtags not matched. - On a clustered server, tags that no server carries make stream creation fail with a JetStream error such
as
no suitable peers for placement. A standalone (non-clustered) server accepts and stores any placement without acting on it.
Taking an existing single-replica stream to three replicas on ssd servers, in two stages:
# Stage 1: the replica count alone. The placement options stay as they were (here unset, so the live # placement is echoed back). Run messenger:setup-transports, then wait until `nats stream info` shows # every replica as current. options: stream_replicas: 3
# Stage 2: the placement. Run messenger:setup-transports again. options: stream_replicas: 3 stream_placement_tags: ['ssd']
Using a NATS Cluster
The transport runs against a NATS cluster as it runs against a single server. A cluster changes how a process
finds the servers, what happens when the one it is connected to goes away, and how many servers a stream lives
on. Two examples show it on the three-server JetStream cluster composer nats:start runs next to the test
servers (examples/README.md describes it):
cluster-failover.php stops the server a worker is connected to, and the
worker carries on through another one; stream-placement.php places a
stream's replicas on tagged servers and stages a move.
framework: messenger: transports: nats_transport: # One server of the cluster, or a name that resolves to several; the client learns the others from it. dsn: 'nats-jetstream+tls://nats.internal:4222/orders/orders.created' options: stream_replicas: 3 # the stream outlives the loss of one server reconnect: true # a worker fails over to another server instead of exiting max_reconnect_attempts: 20 tls_required: true # TLS for the servers learned from the cluster as well tls_ca_file: '/etc/nats/ca.pem'
Tested by:
testReadmeDsnExamplesParseSuccessfully[README: cluster transport],testReadmeClusterOptionsAreAccepted, the examplescluster-failover.phpandstream-placement.php(CI runs both on the test cluster)
Finding the Servers
The DSN names one server. Once connected, the NATS client learns the other servers of the cluster from it (the
connect_urls the server sends) and keeps them for its later dials. What it learns is the address each server
advertises, so behind NAT, in Docker or in Kubernetes set client_advertise in every server's configuration to
an address the application reaches that server on, as the test cluster does. A cluster that advertises nothing
(no_advertise) leaves the client with the DSN's server alone.
Until it has connected once, a process knows only the server its DSN names, and so does every PHP-FPM request
that dispatches, since each one builds its transport anew. One that starts while that server is down cannot
connect, whichever servers of the cluster are up: it fails after one dial, about 6 s against a refused port, or
with reconnect: true after the client's whole attempt loop over that one server. Point the DSN at a name
that resolves to several servers of the cluster (the dial tries each address it resolves to in turn), or at a
load balancer or a Kubernetes Service in front of the cluster.
Losing a Server
What a process does when the server it is connected to goes away depends on
reconnect:
reconnect: false(the default): the operation that runs into the lost connection fails, and the next one dials the next server in the client's list, learned ones included; a server there that is down as well costs that operation a failed dial, and the one after it tries the next. A process that keeps running, such as a long-lived sender, carries on through another server. A consumer worker exits on the failedget(), though, and the worker its supervisor starts knows only the DSN's server again (see above).reconnect: true: the client recovers the connection in the background on a server it learned, and a worker polls on.cluster-failover.phpstops the server a worker is connected to halfway through its messages: the worker receives the rest through another server, without an exception. A handler that fails or succeeds at that moment no longer stops the worker or loses its acknowledgement, in either retry mode (see A Handler Failing During the Loss).
Measured on the test cluster (nats-server 2.15.1, client 2.28.0, PHP 8.5), a stream with three replicas, the
DSN on the server that leads it, that server stopped with docker stop, request_timeout 10 and
max_reconnect_attempts 2 (for the pause, ping_after_idle 1 and 2 s), one transport object throughout:
reconnect: false |
reconnect: true |
|
|---|---|---|
first send(), right after the stop |
fails at once, Reconnect is disabled |
stored at once, through another server (client 2.27.2: 10.0 s, TimeoutException) |
first send(), after a pause longer than ping_after_idle |
stored at once, through another server | stored at once, through another server |
next send() |
stored at once, through another server | stored at once, through another server |
| a new transport whose DSN names the stopped server | 6.0 s, connection refused | 18.4 s, Reconnect attempts exhausted |
Right after the stop, the first send ran into the connection the server had closed. The client sees that close
before it writes (since client 2.28.0): with the option off it reports the loss at once, and the send was not
written; with it on it recovers the connection on another server and the send goes out there. With client 2.27
the send went into the closed socket, and with the option on it waited for a reply that never came. After a pause
longer than ping_after_idle the check that runs first finds the connection gone, in either mode, and the send
goes out through another server (see Losing the Connection). A send the server took and
could not answer before it went away still fails after request_timeout with the option on, unless
resend_on_reconnect sends it again (see Automatic Reconnect); whether such a send was
stored is unknown (see Duplicate Protection).
A Handler Failing During the Loss
The server a worker is connected to goes away while a handler runs, as in a deploy or a rolling restart. The
server closes the connection on its way out, and the client, which reads the socket only while an operation waits
for a reply, has not read that close when the handler returns. What the worker writes next - the ACK of a handled
message, Symfony's retry copy of a failed one, or the NAK of a failed one in nats mode - meets that unread close.
Since client 2.28.0, which 5.8.0 requires, the client notices the close before it writes and handles the write as
one that failed: with reconnect: true the client recovers the connection on another server of the cluster and the
write goes out there, and with reconnect: false it fails at once with Reconnect is disabled.
reconnect: true: the worker carries on, in both retry modes. A handled message's ACK reaches the cluster, and the message is not handled again. A failed message's retry copy (retry_handler: symfony) is stored at once and handled, or its NAK (retry_handler: nats) takes effect and the message comes backnak_delaylater. With client 2.27 the ACK was lost, so the message was handled again afterack_wait, and the retry copy timed out afterrequest_timeoutand stopped the worker.reconnect: false: the worker exits on an ACK or a retry copy, since the client may not reconnect, and the message comes back afterack_wait, to the worker the supervisor starts or to another one, counting one delivery againstmax_deliver. In nats mode a failed message's NAK still keeps the worker going: the NAK checks the connection first with one PING (see Losing the Connection), the transport dials another server, and the NAK goes out there.
The client cannot see a close behind data it has not read yet, such as the close_notify a TLS server sends
ahead of its close (the client's documented limit; not measured here). There the ACK of a handled message is still
taken by the socket and lost, and the retry copy times out as with client 2.27, while the NAK of a failed message
in nats mode is still saved by its PING. A retry copy that went out there, or that the server took and could not
answer before it went away, is sent again on the recovered connection only with
resend_on_reconnect; without it, it times out. Whether such a copy was stored
is unknown (see Duplicate Protection): if it was, and the message delivered again is then
handled, the message is handled twice, once from the original and once from the copy, and if the message
delivered again fails once more, Symfony sends the same copy again. So, on a cluster whose servers restart while
workers run:
- set
deduplicate: truewith astream_duplicate_windowlonger thanack_waitplus the handler's run, so that JetStream drops that second copy; - keep handlers idempotent, for a message handled twice after a lost ACK, or a stored copy next to an original that succeeds, which no id can catch.
Measured on the test cluster (nats-server 2.15.1, client 2.28.0, Symfony Messenger 8.1.10, PHP 8.5) with
Symfony's Worker and retry listener and FrameworkBundle's default retry strategy. The worker's DSN named the
server that leads a stream with three replicas, and the handler stopped that server with docker stop on the
message's first delivery, then failed or returned; request_timeout 10, ack_wait 30, nak_delay 1,
deduplicate: true, each worker run for up to 40 s:
reconnect: false |
reconnect: true |
|
|---|---|---|
retry_handler: symfony, the handler fails |
the worker exits at once, the retry send fails with Reconnect is disabled; a new worker gets the message again 30.0 s after its first delivery |
the worker carries on: the retry copy is stored through another server and handled 0.3 s after the first delivery (client 2.27.2: the worker exited after 10.0 s, TimeoutException) |
retry_handler: nats, the handler fails |
the worker carries on through another server, and gets the message again 1.0 s after the failure | the worker carries on, and gets the message again 1.0 s after the failure |
| either mode, the handler succeeds (an ACK) | the worker exits at once, ack() fails with Reconnect is disabled; a new worker gets the message again 30.0 s after its first delivery and handles it again (client 2.27.2: the ACK was lost and the next pull failed) |
the worker carries on, the ACK reaches the cluster, and the message is not delivered again (client 2.27.2: the ACK was lost, and the message was handled again 30.0 s after its first delivery) |
Nothing was left pending after any run, and with reconnect: false the retry copy was not stored. The
examples handler-failing-during-failover.php and
handler-succeeding-during-failover.php repeat the
reconnect: true column in both retry modes, and CI runs them; both fail with client 2.27.2.
Tested by:
testTheNakOfAFailedDeliveryReachesTheServerThatReplacedOneWhichClosedTheConnectionDuringTheHandler(the real client, against a connection the server closed without the client reading it, on a plain socket and behind unread bytes;reject(), andsend()plusack()),testTheAckOfAHandledMessageReachesTheServerOnTheRecoveredConnectionWithNoCheckFirst,testTheAckOfAHandledMessageIsStillLostToACloseBehindUnreadBytes,testSymfonysRetryCopyAfterTheServerClosedTheConnectionDuringTheHandlerGoesOutOnTheRecoveredConnection,testSettlingAFailedDeliveryChecksTheConnectionFirst, the exampleshandler-failing-during-failover.phpandhandler-succeeding-during-failover.php
Streams on a Cluster
- Replicas. A stream with one replica lives on one server and is unavailable while that server is down.
With
stream_replicas: 3it outlives the loss of one server, with 5 the loss of two. The durable consumer the transport creates takes the stream's replica count. Left unset, the option gives a new stream one replica and leaves an existing stream's count alone (see High Availability). Whensetup()changes the count, it waits out the moment the consumer moves to the stream's new servers (up to 5 seconds). - Placement.
stream_placement_tagsandstream_placement_clusterchoose the servers, and in a super-cluster the cluster, that hold the replicas. A change of the replica count and of the placement has to be staged in two runs ofmessenger:setup-transports. See Stream Placement. - Provisioning.
messenger:setup-transportscan run through any server of the cluster. In a super-cluster a stream withoutstream_placement_clusteris created in the cluster that server belongs to, or in one of its alternates. - Leader changes. A pull that ends because the consumer's leader moved (
409 Leadership Change) is an empty batch in either mode, and the worker pulls on.
TLS on a Cluster
The nats-jetstream+tls:// scheme turns TLS on for the server the DSN names. The servers the client learns
from the cluster are dialled with TLS only when tls_required: true is set as well. Without it, a server that
requires TLS refuses the client (TLS upgrade requested but no TLS context was configured at connect time), so
no failover succeeds, in either mode, and a server that also accepts plain connections would receive the
credentials unencrypted. Set both, as the example above does.
Each dial checks the server's certificate against the host it dialled: the DSN's host for the first dial, the
address a learned server advertises (client_advertise) for the others. Issue each server's certificate for
those names, or set tls_peer_name to a name every server's certificate carries.
Checked by hand on a two-server TLS cluster (nats-server 2.15.1, client 2.27.2): with the
+tlsscheme alone, every dial of the second server failed once the first one stopped, withreconnectoff and on; withtls_required: truethe next dial reached it. A server advertising a name its certificate does not carry was refused (Peer certificate CN=... did not match expected CN=...) untiltls_peer_namenamed one it carries.
Testing
Unit Tests
# Install dependencies composer install # Run static analysis and the fast unit suite after every modification composer test # Run NATS composer nats:start # Run all unit tests with coverage (recommended) composer test:unit # Or run tests manually ./vendor/bin/phpunit # Stopped at random moments, which fails a test that depends on how fast its process runs composer test:unit:paused
The target is to have at least 90% of code coverage.
What's tested:
- DSN parsing and validation
- Configuration option handling
- Authentication: every combination of credentials is refused when the transport is built, or goes through the real client as a CONNECT that carries it (the real logins are in the functional tests below)
- Port configuration
- Error handling
- Interface compliance
Mutation Tests
The unit suite is mutation-tested with Infection to ensure the tests actually detect behavioral changes (not just execute lines):
# Requires a coverage driver (xdebug or pcov) composer test:mutation:canary # mutants that change nothing must all escape composer test:mutation composer test:mutation:check-log # no mutant may be killed by the test harness instead of a test
Configuration lives in infection.json5. It enforces a minimum MSI of 90% and a minimum covered MSI of
95%; the suite scores 98.6% (1298 of 1317 mutants killed, with 100% mutation code coverage).
docs/TESTS.md lists the 19 mutants that survive, with the reason for each:
13 are equivalent, two make no difference with the real client, and three are caught by tests in a child process,
which a mutant does not reach. CI runs all three commands on the PHP 8.5 job and
every day. Scores published before, 100% among them, were not
real: in Infection's mutant processes the client's final classes could not be doubled, so every mutant their
tests covered counted as killed.
Functional Tests
Functional tests require a running NATS server with JetStream enabled:
# Set up functional test dependencies composer test:functional:setup # Start the NATS test servers in Docker, with the three-server cluster the examples use (generates the test # TLS certificates first, which needs openssl, and the test NKey and JWT credentials) composer nats:start # Run functional tests composer test:functional # Stop NATS server composer nats:stop
Manual approach:
# Generate the test TLS certificates, and the NKey and JWT credentials, the servers use (never committed) bash tests/nats/certs/generate.sh bash tests/nats/auth/generate.sh # Set up NATS in Docker (optional) cd tests/nats docker-compose up -d # Run functional tests cd ../functional ./vendor/bin/behat features/ # Stop NATS cd ../nats docker-compose down
What's tested:
- Message publishing
- Message consumption
- Message acknowledgment
- Consumer setup
- Stream persistence
See also: tests/functional/README.md
Advanced Usage
Multiple Transports
Set up multiple independent transports for different use cases:
framework: messenger: transports: # High-priority, low-latency messages nats_fast: dsn: 'nats-jetstream://localhost/fast-stream/fast-topic' options: consumer: 'fast-consumer' batching: 1 # Bulk processing, high throughput nats_bulk: dsn: 'nats-jetstream://localhost/bulk-stream/bulk-topic' options: consumer: 'bulk-consumer' batching: 50 # Audit logging nats_audit: dsn: 'nats-jetstream://localhost/audit-stream/audit-topic' options: consumer: 'audit-consumer' stream_max_age: 2592000 # 30 days stream_replicas: 3
Tested by:
testReadmeDsnExamplesParseSuccessfully[README: fast transport],testReadmeDsnExamplesParseSuccessfully[README: bulk transport],testReadmeDsnExamplesParseSuccessfully[README: audit transport],testReadmeAuditTransportOptionsAreAccepted,testReadmeBatchingExamplesAreAccepted
Multi-Subject Streams
Multiple transports can share the same NATS stream with different subjects. When messenger:setup-transports runs, each transport adds its subject to the existing stream rather than overwriting it:
framework: messenger: transports: # Both transports share the "events" stream nats_orders: dsn: 'nats-jetstream://localhost/events/orders' options: consumer: 'order-consumer' batching: 1 stream_max_age: 300 nats_payments: dsn: 'nats-jetstream://localhost/events/payments' options: consumer: 'payment-consumer' batching: 2
The events stream will have both orders and payments as subjects.
Tested by:
testReadmeDsnExamplesParseSuccessfully[README: multi-subject orders],testReadmeDsnExamplesParseSuccessfully[README: multi-subject payments],testReadmeMultiSubjectOptionsAreAccepted,testSetupUpdatesExistingStreamMergesSubjectsAndPreservesServerConfig, Behat scenarioSetup command merges subjects for transports sharing one stream
Note: When a stream already exists, setup reads the current JetStream configuration, merges in any new subjects, and then overlays the stream settings managed by this transport. Existing subjects are preserved, duplicate subjects are not added, and the existing storage backend is kept for already-created streams.
Provisioning the Stream & Consumer
The stream and its durable pull consumer must exist before messages flow. There are two ways to provision them.
Explicit (default, recommended for production). Provision once via the Symfony command:
framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost/my-stream/my-topic' options: consumer: 'my-consumer'
symfony console messenger:setup-transports nats_transport
This will:
- Create the stream with the configured settings, or update an existing one (merging subjects and keeping the server's own fields)
- Create the consumer with an explicit ACK policy
- Verify that the consumer matches the configuration
Automatic (auto_setup). Set auto_setup=true to have the transport provision the stream and
consumer itself, lazily, on the first send()/get(). Setup then runs once per transport instance, and
again only if JetStream reports the stream or consumer as missing during a pull (for example after NATS
removed an idle consumer that hit inactive_threshold), after close(), or after the transport dialled
again because the connection was lost (see Losing the Connection). It defaults
to false, so unless you opt in, provisioning stays explicit (no hidden stream/consumer creation on
the hot path).
β οΈ Re-provisioning replays the stream. A durable consumer holds the acknowledgement state, so when one is lost that state is gone with it. The replacement is created with
deliver_policy=alland will therefore redeliver every message the stream still retains, including messages that were already acknowledged. Under the defaultlimitsretention that means the whole stream, so make your handlers idempotent before combiningauto_setupwith a shortinactive_threshold.workqueueretention does not have this problem, because acknowledged messages are removed from the stream. This is inherent to losing a durable consumer, not toauto_setup: a restarted worker rebuilds the consumer the same way.
framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost/my-stream/my-topic?auto_setup=true'
Note on retention & storage:
stream_retentionandstream_storageare fixed when the stream is created - NATS rejects changing either on an existing stream. On the update path the transport preserves the live server values, so changing these options on an already-created stream is a no-op. To change retention or storage, recreate the stream: delete it, then re-runmessenger:setup-transports. Withauto_setup=truea running worker recreates it on its next pull: deleting the stream deletes its consumers with it, so the pull fails with status 503 (nothing is left to answer it), which is one of the statuses that trigger re-provisioning.
Note on a server that stops: a pull the server ended for its own shutdown (
409 Server Shutdown) or for a leadership change (409 Leadership Change) is an empty batch, in either mode, and not one of the statuses that trigger re-provisioning: withauto_setupthe transport does not runsetup()against a server that is going away. After a shutdown the next operation checks the connection first. Every other 409,Consumer Deletedamong them, still triggers one re-provisioning attempt, and the 409s that report configuration (Consumer is push based,Exceeded MaxRequestBatch) still propagate. A pull written while a server is going away gets 503, as the server has stopped the consumer and not yet closed the connection: in the last moments of a graceful stop (NATS 2.15 stops each durable consumer on a file stream in about 10 ms), and for the whole grace period of lame duck mode, in which the server stops JetStream at once and keeps the connections open (10 s by default). A missing consumer gives the same 503, so after a 503 the transport asks the client why (JetStreamContext::diagnoseNoResponders(), client 2.29), in either mode; the client asks for the consumer once (CONSUMER.INFO). A server that stays answers at once, with the consumer or with 404, and the 503 keeps the reading above:auto_setupprovisions again, and without it the 503 propagates. A server that is going away drops the reply, and closes the connection or announces lame duck mode: that pull is an empty batch, and the next operation checks the connection first. The client stops waiting once the connection is closed, being recovered or replaced by a reconnect, or its server announced lame duck mode, where it would otherwise keep the question waiting untilrequest_timeout; once the server has announced lame duck mode it asks nothing more. A question that goes unanswered on a connection that stays, as for a user allowed to pull but not to ask for consumer info ($JS.API.CONSUMER.INFO.<stream>.<consumer>), keeps the 503's reading, afterrequest_timeout: a missing consumer is not hidden behind empty pulls.Tested by:
testA503FromAServerGoingAwayIsAnEmptyBatch,testTheOperationAfterAServerGoingAwayChecksTheConnectionTheClientHas,testA503KeepsItsReadingWhenTheServerAnswers,testA503KeepsItsReadingWhenNothingAnswersOnAConnectionThatStays,testWithTheRealClientAPullOnAStoppingServerIsAnEmptyBatchAndTheNextOneDialsAgain,testWithTheRealClientAndReconnectAPullOnAStoppingServerIsAnEmptyBatchWithoutWaitingForTheRequestTimeout,testWithTheRealClientAPullOnAServerInLameDuckModeIsAnEmptyBatchWhileTheConnectionStays, Behat scenariosA worker keeps consuming across a server restartandA worker keeps consuming across a server that leaves in lame duck mode
Note on
replay_policy: the replay policy of a durable consumer is fixed when the consumer is created. NATS rejects an update that changes it, in both directions: once a consumer exists asoriginal, removing the option again does not restoreinstanteither, because the server reads the omitted field as a change back to its default. The transport therefore always writes the existing consumer's own replay policy and applies the configured one only to a consumer that does not exist yet. Changing it on a running deployment is a no-op until you delete the consumer and let setup recreate it.
Note on the deny flags:
stream_deny_deleteandstream_deny_purgecan be switched on but never off. NATS rejects an update that cancels either one, so on a stream that already denies deletes or purges the transport keeps the server's value and afalsein your configuration is ignored. Recreate the stream to lift a deny.stream_allow_directandstream_allow_rollup_headersare freely mutable in both directions.
Note on
stream_max_consumers: NATS servers up to and including 2.11 also refuse to changemax_consumerson an existing stream. The transport therefore writes it only when you set the option explicitly; when you leave it unset, the stream keeps whatever value it already has. Setting it on an already-created stream works on NATS 2.12 and newer, and is rejected by the server on older versions - recreate the stream to change it there.
Note on placement:
stream_placement_clusterandstream_placement_tagsfollow the same rule asstream_max_consumers: they are written only when you set them, and unset options leave the stream's existing placement in place. A JetStream update that omitsplacementwould clear it, so the transport echoes the live value back whenever the options are unset, and a set option replaces only its own field of the live placement. NATS rejects a placement change combined with astream_replicaschange on an existing stream, and withauto_setupeverysend()andget()repeats that rejected update until the two are staged. See Stream Placement for the steps.
Tested by:
testSetupCreatesStreamAndConsumer,testSetupPassesConfiguredStreamOptions,testSetupPassesNewStreamPolicyOptions,testSetupPassesNewConsumerOptions,testSetupPassesStreamPlacementOnCreate,testSetupUpdateAppliesConfiguredStreamPlacementToAnExistingStream,testSetupUpdatePreservesServerStreamPlacementWhenNotConfigured,testAutoSetupProvisionsOnFirstSendOnce,testAutoSetupProvisionsOnFirstGet,testAutoSetupDisabledByDefaultDoesNotProvisionOnSend,testSetupUpdatesExistingStreamMergesSubjectsAndPreservesServerConfig, Behat scenariosSetup NATS stream with max age configuration,Setup command handles existing streams gracefully, andCustom consumer name is registered in JetStream
Delayed / Scheduled Messages
Requires NATS Server >= 2.12 with JetStream enabled. An older server has no message schedules and ignores the
setting without a word, so the transport reads the server's version: messenger:setup-transports fails there,
before it creates anything, and so does the send of a delayed message, which that server would store and never
deliver. A message without a delay is sent as always.
The 'scheduled_messages' option requires NATS Server >= 2.12, but the connected server reports 2.10.29. Disable scheduled_messages or upgrade NATS.
Tested by:
testSetupRefusesScheduledMessagesOnAServerWithoutSchedules,testSendRefusesToScheduleOnAServerWithoutSchedules,testSetupWithScheduledMessagesGoesOnWhenTheServerMaySchedule,testSendSchedulesOnAServerThatMaySchedule,testSetupWithoutScheduledMessagesDoesNotLookAtTheServersVersion,testSendWithoutAScheduleDoesNotLookAtTheServersVersion,testSetupConnectsBeforeItLooksAtTheServersVersion,testTheFirstDelayedSendConnectsBeforeItLooksAtTheServersVersion, the exampledelayed-messages.php, which CI runs on the latest and on the oldest supported server
Enable scheduled_messages in the DSN to use Symfony's DelayStamp for deferred delivery:
framework: messenger: transports: nats_transport: dsn: 'nats-jetstream://localhost/my-stream/my-topic?scheduled_messages=true'
Then dispatch messages with a delay:
use Symfony\Component\Messenger\Stamp\DelayStamp; // Deliver after 30 seconds $bus->dispatch(new MyMessage(), [new DelayStamp(30000)]);
Tested by:
testSendWithDelayStampPublishesToDelayedSubjectWithScheduleHeaders,testSendDelayedMessageSchedulesAtRequestedDelay,testSendDelayedMessageNeverSchedulesBeforeRequestedDelay,testReadmeScheduledMessagesDsnEnablesFeature, Behat scenariosDelayed messages are delivered after the scheduled timeandDelayed messages are not available to the consumer before the scheduled time
When scheduled_messages is enabled and a DelayStamp is present:
- The message is published to a
{topic}.delayed.{uuid}subject withNats-ScheduleandNats-Schedule-Targetheaders - The stream is created with an additional
{topic}.delayed.>subject andallow_msg_schedulesenabled - NATS holds the message and delivers it to the original topic at the scheduled time
- The consumer processes it like any other message
The DelayStamp delay (milliseconds) maps onto a NATS @at schedule, which has whole-second
resolution. The delay is rounded up to the next whole second, so a message is never delivered
before the requested delay elapses (it may arrive up to ~1 second later); a sub-second delay therefore
schedules at the next whole second rather than firing immediately.
When scheduled_messages is disabled (the default), any DelayStamp on the envelope is silently ignored and messages are published immediately. That includes the DelayStamp Symfony's retry adds, so retries run back to back, unless the transport has retry_via: nak (see Retry Handler Behavior).
Stream Monitoring
View stream and consumer information:
# List streams nats stream list # View stream info nats stream info my-stream # List consumers nats consumer list my-stream # View consumer info nats consumer info my-stream my-consumer # View message count nats consumer info my-stream my-consumer --json | jq '.state.num_pending'
Manual Message Operations
use Symfony\Component\Messenger\MessageBusInterface; use Symfony\Component\Messenger\Transport\TransportInterface; // Get message count $count = $transport->getMessageCount(); // Check if messages are pending if ($count > 0) { echo "Pending messages: $count"; }
getMessageCount(), which messenger:stats shows, returns 0 when NATS cannot be reached or refuses the login, the
same as for an empty queue, so during an outage, or with a wrong password or seed, the queue reads as empty there, where
send() and get() raise the error. Without reconnect: true it gives up after a single connection attempt (about 6
seconds for a refused connection, see Connection Timeout). With it the client's attempt loop runs
first: about 109 seconds for a refused connection at the default attempts, and about 43 seconds for a seed on a server that
sends no nonce. A login the server itself refuses returns at once in both modes. Monitor the connection itself, or the
stream with the nats CLI, rather than reading a 0 as "nothing to do".
Nothing waits for a plain ACK, so for a moment after ack() returned the count can still include that message. With
ack_sync the server has applied the acknowledgement when ack() returns.
Tested by:
testGetMessageCountReturnsConsumerPendingMessages,testGetMessageCountFallsBackToStreamState,testGetMessageCountReturnsZeroWhenLookupsFail,testGetMessageCountSumsAckPendingAndPending,testMessageCountThatCannotConnectDialsOnceAndReturnsZero,testMessageCountOfALoginTheServerRefusesDialsOnceAndReturnsZero, the Behat scenariosA seed the server does not know is refused by the serverandA JWT with another user's seed is refused by the server. The timings were measured by hand.
Troubleshooting
Connection Issues
Error: "Connection refused"
# Check NATS is running nats-server --js # Verify host and port nats-jetstream://localhost:4222/stream/topic
Tested by: Behat scenario
Setup command fails gracefully when NATS is unavailable
Error: "Stream not found"
# Run setup command to create stream
symfony console messenger:setup-transports nats_transport
Tested by:
testSetupCreatesStreamAndConsumer,testSetupUpdatesStreamWhenItAlreadyExists, Behat scenariosSetup NATS stream with max age configurationandSetup command handles existing streams gracefully
Error: "Cannot pull from NATS stream ...: the subscription limit was exceeded"
The connection may hold only one subscription, and the transport needs two. Raise the server's max_subscriptions,
or the account's or user's subscription limit, to at least two (see Subscription Limits).
Tested by:
testAtALimitOfOneThePullFailsWithATransportExceptionNamingTheLimit,testAtALimitOfTwoPullsDeliverAndAcknowledge
Error: "Reconnect attempts exhausted"
With reconnect: true the client's recovery of a lost connection gave up after
max_reconnect_attempts, and the operation that reports it ran the client's attempt loop inside the call. The
outage outlasted one recovery cycle: raise max_reconnect_attempts for longer outages (the client's default of
10 covers about 109 s against a server that refuses connections), or leave the option off to fail fast and let
the process supervisor restart the worker. The logger shows the attempts at warning level and, when
acknowledgements were buffered during the outage, the discard at error level (see Logging).
With a seed on a server that sends no nonce it is not an outage, and raising the option only makes it take longer: the client
retried a login that cannot work, and the cause is the previous exception, NKey authentication requires server nonce from INFO. See NKey Authentication.
Tested by:
testAReconnectThatGivesUpWithABufferedAcknowledgementIsReportedToTheLoggerAndTheNextOperationFails
Error: "TLS upgrade requested but no TLS context was configured at connect time"
On a cluster that requires TLS, the client dialled a server it learned from the cluster without TLS, because
only the nats-jetstream+tls:// scheme asked for it. Set tls_required: true as well (see
TLS on a Cluster).
Error: "replicas > 1 not supported in non-clustered mode"
stream_replicas above 1 needs a JetStream cluster; on a standalone server leave it at 1 (see
High Availability).
Message Processing Issues
Messages not being consumed
# Check consumer exists nats consumer list my-stream # View consumer status nats consumer info my-stream my-consumer # Check for errors in consumer nats consumer info my-stream my-consumer --json | jq '.state'
Messages stuck in pending
# Check handler is not throwing exceptions # Verify handler implementation # Check application logs for errors
Architecture
The bridge consists of two main components, wired into the framework by a thin bundle:
IDCTNatsMessengerBundle
- Registers
NatsTransportFactoryand theIgbinarySerializerservice in the application container - Hands the application's
loggerservice to the factory, under Monolog'smessengerchannel - Auto-enabled by Symfony Flex; no semantic configuration
NatsTransportFactory
- Handles DSN scheme detection (
nats-jetstream://) - Creates
NatsTransportinstances, handing them its optional PSR-3 logger for the NATS client and the transport's warning in nats mode - Validates configuration
NatsTransport
- Implements Symfony's
TransportInterface,MessageCountAwareInterface,SetupableTransportInterface,KeepaliveReceiverInterface, andCloseableTransportInterface - Manages stream and consumer connections: dials again itself after a lost connection, or leaves the recovery to
the NATS client with
reconnect: true - Handles message serialization via a pluggable
SerializerInterface(igbinary when constructed directly without one) - Supports batching and explicit acknowledgment
Performance Tips
-
Choose appropriate batching
- Start with
batching: 5for balanced performance - Increase to 20+ for high throughput workloads
- Use 1 for strict low-latency requirements
- Start with
-
Set reasonable timeouts
max_batch_timeout: 0.5for responsive systemsmax_batch_timeout: 2.0for background jobsconnection_timeout: 1.0for local/regional deploymentsconnection_timeout: 3.0+for cross-region or high-latency networks
-
Use appropriate replicas
stream_replicas: 1for developmentstream_replicas: 3for production, on a JetStream cluster (see Using a NATS Cluster)
-
Monitor performance
- Use
getMessageCount()to track queue depth - Monitor handler execution time
- Watch for stuck messages
- Use
Security Considerations
β οΈ Deserialization of Untrusted Data
The default IgbinarySerializer (and any serializer extending AbstractEnveloperSerializer) deserializes raw message payloads from NATS into PHP objects. PHP object unserialization is a well-known attack vector - a crafted payload can trigger arbitrary code execution via magic methods (__wakeup, __destruct, etc.).
β οΈ PhpSerializer fallback: when no serializer is configured and
ext-igbinaryis not installed, the transport automatically falls back to Symfony'sPhpSerializer, which uses nativeunserialize()- the same untrusted-deserialization (object injection) risk as igbinary, not a safer alternative. The transport emits anE_USER_WARNINGwhen this happens. Do not rely on the fallback in production: either installext-igbinaryor explicitly configure a serializer (ideally a safe-format one, per below).
If your NATS topics are not fully trusted (e.g. shared infrastructure, external publishers), you should:
- Implement a custom serializer that uses a safe format (JSON, Protobuf) instead of PHP object serialization
- Add message-level authentication (e.g. HMAC signatures) to verify publisher identity before deserializing
- Restrict NATS topic publish permissions via ACLs so only trusted services can publish
The type check (instanceof Envelope) happens after deserialization, which is too late to prevent exploitation.
Stream-Exists Detection During Setup
During setup(), if createStream fails the transport detects a pre-existing stream deterministically by querying JetStream stream info: a 404 means the stream is absent (so the creation error was genuine and is rethrown), while a successful lookup means the stream exists (so it is updated, reusing the fetched configuration). This relies on the JetStream stream-info API rather than matching server-specific conflict strings ("already in use" / "already exists"), whose wording varies across NATS versions.
If you experience unexpected behavior during stream setup, confirm the stream can be queried via JetStream stream-info APIs and review the exact error returned by your NATS server version.
Publish Response Validation
On send(), the transport awaits the JetStream publish acknowledgement returned by the client's publish() call. The client validates that acknowledgement and raises a JetStreamException if JetStream reports an error or returns an empty/malformed response, so a proxy or protocol misconfiguration fails closed instead of silently accepting an invalid publish acknowledgement.
General Recommendations
-
Authentication
- Prefer environment variables or explicit options for credentials over hard-coded DSNs
- If you use credentials in a DSN, avoid logging the full DSN because it may expose secrets
- Store credentials in environment variables
- Never commit credentials to version control
-
Message Encryption
- Encrypt sensitive data before dispatching
- NATS can be configured with TLS for transit encryption
- Implement application-level encryption for sensitive payloads
-
Access Control
- Restrict stream/consumer creation to authorized users
- Use NATS access control lists (ACLs) for fine-grained permissions
- Audit stream operations
Contributing
Contributions are welcome! Please ensure:
- Every modification runs the relevant verification commands before it is considered done
- Minimum verification for PHP changes:
composer test - All tests pass:
composer test:unit - Code coverage remains above 90%
- New features include corresponding tests
- Documentation is updated
- Functional tests pass:
composer test:functional(if applicable) docs/TESTS.mdis kept up to date when tests are added, removed, or renamed- Each release has an entry in
docs/CHANGELOG.mdfollowing Keep a Changelog format - When a PR is merged or its features are adapted, a description is added to
docs/PRs/
Quick Development Workflow
# 1. Run static analysis and the default unit suite after each modification composer test # 2. Set up functional tests (first time only) composer test:functional:setup # 3. Start NATS for functional tests composer nats:start # 4. Run functional tests composer test:functional # 5. Clean up composer nats:stop
License
MIT License - see LICENSE file for details
Support
For issues, questions, or suggestions:
- Check the troubleshooting section
- Check existing issues on GitHub
- Create a new issue with detailed information