ostark / craft-async-queue
A queue handler that moves queue execution to a non-blocking background process
Installs: 266 614
Dependents: 11
Suggesters: 0
Security: 0
Stars: 93
Watchers: 2
Forks: 7
Open Issues: 4
Type:craft-plugin
Requires
- php: ^8.0
- craftcms/cms: ^5.0
- symfony/process: ^5.0|^6.0
Requires (Dev)
- craftcms/phpstan: *
- craftcms/rector: dev-main
- phpunit/phpunit: ^8.0|^9.0
This package is auto-updated.
Last update: 2024-10-19 08:49:45 UTC
README
With Craft's job queue you can run heavy tasks in the background. Unfortunately, this is not entirely true. When runQueueAutomatically => true
(default), the job queue is handled by a ajax (FPM) call.
With many jobs in the queue and limited PHP-FPM processes this can break your site. This plugin replaces Craft's default queue handler and moves queue execution to a non-blocking background process. The command craft queue/run
gets executed right after you push a job to the queue.
Here you can find the initial discussion I started at craftcms/cms
.
Sponsor 🐇
Development happens in my free time, but also during working hours. Thanks fortrabbit.com!
Licence 🌳
This package is Treeware. If you use it in production, then we ask that you buy the world a tree to thank me for my work. By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats.
Requirements
- Craft 3 or 4
- Permissions to execute a php binary
- proc_open()
- PHP >=7.1 (for PHP 7.0 use
ostark/craft-async-queue:1.3.*
)
Installation
cd your/craft-project
composer require ostark/craft-async-queue
php craft install/plugin async-queue
If you run into Composer version conflicts:
composer config platform --unset
composer update
php craft migrate/all
composer require ostark/craft-async-queue
php craft install/plugin async-queue
Configuration (optional)
The plugin uses symfony/process to execute the php
binary. Usually the binary is located in /usr/bin/
, but other common locations are auto detected as well. With the ENV var PHP_BINARY
you can explicitly set the path, e.g. in your .env file like this:
PHP_BINARY="/usr/local/Cellar/php71/7.1.0_11/bin/php"
By default 1
background process handles the queue. With the ASYNC_QUEUE_CONCURRENCY
ENV var you can modify this behaviour.
# No concurrency
ASYNC_QUEUE_CONCURRENCY=1
# Or max 5 background processes
ASYNC_QUEUE_CONCURRENCY=5
To disable the plugin in certain environments, like on Windows which is not supported yet, set the DISABLE_ASYNC_QUEUE
ENV var.
DISABLE_ASYNC_QUEUE=1
Note that the plugin will overwrite the runQueueAutomatically
setting with Craft CMS. No configurartion required.
Tests
Beside the test suite you can run from the command line with this shortcut: composer tests
, you can perform a test in the Craft CP.
Navigate to Utilities
> Async Queue Test
and hit the Run test
button.
Events
The command that runs in the background is basically php craft queue/run
, however we add some linux specific syntax that executes the command in a non-blocking way.
By setting useDefaultDecoration
to false
you prevent this. You have also the ability to modify the command itself.
// Add handler
\yii\base\Event::on(
\ostark\AsyncQueue\QueueCommand::class,
\ostark\AsyncQueue\QueueCommand::EVENT_PREPARE_COMMAND,
function(\ostark\AsyncQueue\Events\QueueCommandEvent $event) {
$event->useDefaultDecoration = false;
$event->commandLine = "BEFORE {$event->commandLine} AFTER";
}
);
Under the hood: Process list
Empty queue (only php-fpm master is running)
$ ps auxf | grep php
root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf)
New job pushed (php-fpm master + child + /usr/bin/php daemon started)
$ ps auxf | grep php
root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf)
app 3031 2.2 0.2 718520 45992 ? S 12:31 0:00 \_ php-fpm: pool www
app 3033 1.2 0.2 280936 32808 ? S 12:31 0:00 /usr/bin/php craft queue/run
app 3034 0.0 0.0 4460 784 ? S 12:31 0:00 \_ sh -c /usr/bin/php craft queue/exec "1234" "0" "1"
app 3035 1.2 0.2 280928 32280 ? S 12:31 0:00 \_ /usr/bin/php craft queue/exec 1234 0 1