lukewaite / laravel-queue-aws-batch
Laravel Queue for AWS Batch, enabling users to submit jobs for processing to a Batch queue.
Installs: 22 111
Dependents: 0
Suggesters: 0
Security: 0
Stars: 8
Watchers: 4
Forks: 16
Open Issues: 3
Requires
- aws/aws-sdk-php: ^3.20.6
- illuminate/support: >5.4
Requires (Dev)
- laravel/framework: >5.4
- mockery/mockery: ^0.9.6
- phpunit/phpunit: ~4.8
- scrutinizer/ocular: ^1.3
- squizlabs/php_codesniffer: ^2.8
README
Supported Versions
Installation
See the table above for package version information, and change the version below accordingly.
Using composer
, run:
composer require lukewaite/laravel-queue-aws-batch ~2.0
Usage
-
Your Laravel application will need to be dockerized and pushed into a container registry of your choice. The
ENTRYPOINT
should be set toartisan
. -
Add a new queue to your
config/queues.php
config file'sconnections
array:
[
'batch' => [
'driver' => 'batch',
'table' => 'jobs',
'queue' => 'first-run-job-queue',
'jobDefinition' => 'my-job-definition',
'expire' => 60,
'region' => 'us-east-1'
]
]
This queue transport depends on being able to write it's queue jobs to a database queue. In this example, it writes it's
jobs to the jobs
table. You'll need to use the artisan queue:table
to create a migration to create this table.
-
Create an AWS Batch job queue with the same name as the
queue
config setting. This is where the Batch connector will push your jobs into Batch. In this case, my queue name would befirst-run-job-queue
. -
Create a AWS Batch job definition for each queue you define that looks something like this:
{ "jobDefinitionName": "my-laravel-application", "type": "container", "parameters": {}, "retryStrategy": { "attempts": 10 }, "containerProperties": { "image": "<your docker image>", "vcpus": 1, "memory": 256, "command": [ "queue:work-batch", "Ref::jobId", "--tries=3" ], "volumes": [], "environment": [], "mountPoints": [], "ulimits": [] } }
Here, you configure your container to start, run the queue:work-batch
command (assuming artisan
is your entrypoint)
and pass in the name of the queue, first-run-job-queue
as well as the Ref::jobId
param, which is passed in when
the Batch connector creates the job.
It is important that you configure a retryStrategy with more "attempts" than you are running tries
if you provide that
argument. Otherwise, Batch will not retry your job if it fails. Laravel 5.1 does not write to the failed job queue until
the next run after tries has been exceeded by jobs failing. Newer versions will write to the queue in the same run, so
this requirement can be relaxed later.
- Add the Service Provider to your application:
- In
config/app.php
add to theproviders
array:LukeWaite\LaravelQueueAwsBatch\BatchQueueServiceProvider::class
- In
Limitations
Delayed Jobs
AWS Batch has no method to delay a job and as it's our runner, we don't have an easy work around. If you require delayed jobs for your use case, at this point my recommendation would be to use a regular DB queue, and to fire a job into it which will fire your batch job at the correct time.