amranidev / micro-bus
Build your laravel/lumen microservice application with AWS SNS/SQS
Installs: 22 866
Dependents: 0
Suggesters: 0
Security: 0
Stars: 73
Watchers: 5
Forks: 11
Open Issues: 0
Type:laravel-package
Requires
- aws/aws-sdk-php: ^3.95
Requires (Dev)
- orchestra/testbench: ^7.5
README
Build your laravel microservices with micro-bus
What is micro-bus?
MicroBus is a laravel/lumen package for building microservices with event-driven architecture (Pub-Sub) using Amazon web services (SNS/SQS).
What is event-driven microservices?
-
Edmunds: Event-Driven, Serverless, and Cost-Effective Enterprise Message Bus
-
GOTO 2017 • The Many Meanings of Event-Driven Architecture • Martin Fowler
-
GOTO 2018 • Pragmatic Event-Driven Microservices • Allard Buijze
-
Building Scalable Applications and Microservices: Adding Messaging to Your Toolbox
Installation.
Laravel.
-
Install the package,
composer require amranidev/micro-bus
. -
Publish the subscriber config file,
php artisan vendor:publish --tag=subscriber
. -
Publish the publisher config file,
php artisan vendor:publish --tag=publisher
. -
Add the subscriber and the publisher environment variables.
- In the
.env
file add.
# Standard SQS keys. SUBSCRIBER_SQS_KEY=<SQS-KEY-AWS> SUBSCRIBER_SQS_SECRET=<SQS-SECRET-AWS> SUBSCRIBER_SQS_PREFIX=https://sqs.<sqs-region>.amazonaws.com/<project-id> SUBSCRIBER_SQS_QUEUE=<QUEUE-NAME-AWS> SUBSCRIBER_SQS_REGION=<SQS-REGION-AWS> PUBLISHER_SNS_KEY=<SNS-KEY-AWS> PUBLISHER_SNS_SECRET=<SNS-KEY-AWS> PUBLISHER_SNS_REGION=<SNS-REGION-AWS> # FIFO SQS Keys. SUBSCRIBER_FIFO_SQS_KEY=<SQS-KEY-AWS> SUBSCRIBER_FIFO_SQS_SECRET=<SQS-SECRET-AWS> SUBSCRIBER_FIFO_SQS_PREFIX=https://sqs.<sqs-region>.amazonaws.com/<project-id> SUBSCRIBER_FIFO_SQS_QUEUE=<QUEUE-NAME-AWS> SUBSCRIBER_FIFO_SQS_REGION=<SQS-REGION-AWS> # SNS Keys. PUBLISHER_SNS_KEY=<SNS-KEY-AWS> PUBLISHER_SNS_SECRET=<SNS-KEY-AWS> PUBLISHER_SNS_REGION=<SNS-REGION-AWS>
- In the
-
Add the Queue connection configuration in
config/queue.php
.// Standard SQS configuration. 'subscriber' => [ 'driver' => 'subscriber', 'key' => env('SUBSCRIBER_SQS_KEY', 'your-public-key'), 'secret' => env('SUBSCRIBER_SQS_SECRET', 'your-secret-key'), 'prefix' => env('SUBSCRIBER_SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'), 'queue' => env('SUBSCRIBER_SQS_QUEUE', 'your-queue-name'), 'region' => env('SUBSCRIBER_SQS_REGION', 'us-east-1'), 'retry_after' => 90, ], // FIFO SQS configuration. 'subscriber-fifo' => [ 'driver' => 'subscriber', 'key' => env('SUBSCRIBER_FIFO_SQS_KEY', 'your-public-key'), 'secret' => env('SUBSCRIBER_FIFO_SQS_SECRET', 'your-secret-key'), 'prefix' => env('SUBSCRIBER_FIFO_SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'), 'queue' => env('SUBSCRIBER_FIFO_SQS_QUEUE', 'your-queue-name'), 'region' => env('SUBSCRIBER_FIFO_SQS_REGION', 'us-east-1'), 'retry_after' => 90, ],
Congratulations you have successfully installed micro-bus 🚀
Lumen.
-
Install the package,
composer require amranidev/micro-bus
. -
Add the subscriber and the publisher environment variables.
- In the
.env
file add.
SUBSCRIBER_SQS_KEY=<SQS-KEY-AWS> SUBSCRIBER_SQS_SECRET=<SQS-SECRET-AWS> SUBSCRIBER_SQS_PREFIX=https://sqs.<sqs-region>.amazonaws.com/<project-id> SUBSCRIBER_SQS_QUEUE=<QUEUE-NAME-AWS> SUBSCRIBER_SQS_REGION=<SQS-REGION-AWS> PUBLISHER_SNS_KEY=<SNS-KEY-AWS> PUBLISHER_SNS_SECRET=<SNS-KEY-AWS> PUBLISHER_SNS_REGION=<SNS-REGION-AWS>
- In the
-
Create
config
folder in the root directory. -
Create
subscriber.php
in the config folder.
<?php return [ 'subscribers' => [ '__CLASSNAME__' => 'TopicArn' ] ];
- Create
publisher.php
in the config folder.
<?php return [ 'sns' => [ 'key' => env('PUBLISHER_SNS_KEY'), 'secret' => env('PUBLISHER_SNS_SECRET'), 'region' => env('PUBLISHER_SNS_REGION'), ], 'events' => [ 'user_created' => 'arn:aws:sns:eu-west-1:111111111111:user_created' ] ];
- Create
queue.php
in the config folder.
Copy the same queue.php
from laravel/laravel and add the subscriber configuration into connections
.
'subscriber' => [ 'driver' => 'subscriber', 'key' => env('SUBSCRIBER_SQS_KEY', 'your-public-key'), 'secret' => env('SUBSCRIBER_SQS_SECRET', 'your-secret-key'), 'prefix' => env('SUBSCRIBER_SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'), 'queue' => env('SUBSCRIBER_SQS_QUEUE', 'your-queue-name'), 'region' => env('SUBSCRIBER_SQS_REGION', 'us-east-1'), 'retry_after' => 90, ],
- Configure
subscriber
,publisher
,queue
and register the ServiceProvider inbootstrap/app.php
.
$app->configure('publisher') $app->configure('subscriber'); $app->configure('queue'); $app->register(Amranidev\MicroBus\MicroBusServiceProvider::class);
Congratulations you have successfully installed micro-bus in lumen 🚀
Usage.
Note that you need to define your Topics in SNS, create the queues in SQS and tie SNS topics to the queues, AWS configuration is not included in this documentation.
After installing the package in both laravel nodes, now, let's make them talk to each other. Basically, they can be both subscribers or publishers at the same time, or, you can make one as subscriber and the other as publisher, it is entirely up to you.
Let's say we have a laravel app called A and a lumen microservice called B.
When a user is created in the A, B needs to add the user's email to the mailing list,
In this case A is the publisher and B is the subscriber.
Note that the SNS configuration needs to be added to A and the SQS needs to be added in B, see installation steps above.
Setup Laravel app (Microservice A).
First of all, let's add the topic that we've created in AWS to A,
we need to specify an event name user_created
and the AWS TopicArn in config/publisher.php
<?php return [ 'sns' => [ 'key' => env('PUBLISHER_SNS_KEY'), 'secret' => env('PUBLISHER_SNS_SECRET'), 'region' => env('PUBLISHER_SNS_REGION'), ], 'events' => [ 'user_created' => 'arn:aws:sns:eu-west-1:111111111111:user_created' ] ];
After we configured the SNS in A, we need to add the functionality, when a user is created, we need to publish a message to SNS, to do so:
... Publisher::publish('user_created', $user); ...
Alternatively you can publish messages with:
- Service container:
// Standard SNS. app('sns.connection')->publish('event', $data); // Fifo SNS. app('sns.fifo.connector')->publish('event', $data);
- Artisan command:
php artisan bus:publish <data> <eventOrTopicname>
Set up Lumen (Microservice B).
Now, let's create the subscriber class which will be listening for A in B.
php artisan make:subscriber UserCreated
This command will scaffold a job class for you in app/Subscribers/
.
<?php namespace App\Subscribers; use Amranidev\MicroBus\Sqs\Traits\JobHandler; class UserCreated { use JobHandler; /** * @var mixed */ public $payload; /** * @var \Illuminate\Queue\Jobs\Job */ public $job; /** * Execute the job. * * @return void */ public function handle() { // } }
As you can see, the job created has $payload
property, it
is simply the piece of data that A will be publishing, in our case will be the $user
.
The Handle method is the one responsible for executing the job coming form SQS,
/** * Execute the job. * * @return void */ public function handle(MailingList $mailingList) { $user = $this->payload; // MailingList is resolved automatically from the container. $mailingList->addUser($user->name, $user->email); }
The last thing we need to do in B is to tie this class to the TopicArn in config/subscriber.php
.
<?php return [ 'subscribers' => [ \App\Subscribers\UserCreated::class => 'arn:aws:sns:eu-west-1:111111111111:user_created' ] ];
And run the queue:work
command, php artisan queue:work <connection-name>
,
Contributing.
Thank you for considering contributing to this project! The contribution guide can be found in Contribution guide.
Alternative (Google Cloud Platform).
Testing.
- Pull the localstack/localstack docker image.
docker pull localstack/localstack
Create a docker-compose.yml
file.
version: "3.8" services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}" image: localstack/localstack network_mode: bridge ports: - '4566-4597:4566-4597' environment: - SERVICES=sqs,sns - DEBUG=${DEBUG-} - DATA_DIR=${DATA_DIR-} - LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR-} - LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY-} # only required for Pro - HOST_TMP_FOLDER=${TMPDIR:-/tmp/}localstack - DOCKER_HOST=unix:///var/run/docker.sock volumes: - "${TMPDIR:-/tmp}/localstack:/tmp/localstack" - "/var/run/docker.sock:/var/run/docker.sock"
-
Run
docker-compose run
to start localstack, if you're using mac runTMPDIR=/private$TMPDIR docker-compose up
. -
Finally run
phpunit
.