ibrahimgunduz34/shepherd

Process pooling component for Symfony Process component

dev-master 2019-03-02 18:21 UTC

Requires

• psr/log: 1.1.0
• squizlabs/php_codesniffer: ^3.4
• symfony/config: ^3.3|^4.0
• symfony/dependency-injection: ^3.3|^4.0
• symfony/http-kernel: ^3.3|^4.0
• symfony/process: ^3.3|^4.0

Requires (Dev)

• phpunit/phpunit: ^5.7|^7.5.0

proprietary

• Ibrahim Gunduz

README

Shepherd is an easy way to run the processes in parallel. Also, you can control how many processes you can run in parallel or decide to stop all jobs in the queue if anyone is failed.

How To Install

Call the following command to add the package to your project as a composer dependency.

composer require ibrahimgunduz34/shepherd

How To Configure

You can simply define your process pools in pools section separately. Also, you can define some default options in defaults section and override them in each pool definition if it's needed.

shepherd:
    defaults:
        max_processes: 4
        fail_on_error: true
    pools:
        foo:
            max_processes: 2
            fail_on_error: false
        bar:
            max_processes: 8

How To Use

Basically, Shepherd creates services for each pool definition. So you can simply inject the pools anywhere. It creates services withshepherd.pool.<pool name> naming convention.

App\Service\MyService:
    class: App\Service\MyService
    arguments:
        - '@shepherd.pool.foo'

You can keep adding new processes until starting the pool processing.

<?php
namespace App\Service;

use Shepherd\Bundle\ProcessPoolBundle\ProcessPool;
use Symfony\Component\Process\Process;

class MyService 
{
    /** @var ProcessPool */
    private $pool;
    
    function __construct(ProcessPool $pool) {
        $this->pool = $pool; 
    }
    
    public function performSomething() {
        //...
        $this->pool->append(new Process([__DIR__ . '../bin/console', 'do:something', 'param1', 'param2']));
        $this->pool->append(new Process([__DIR__ . '../bin/console', 'do:something', 'param3', 'param4']));
        $this->pool->append(new Process([__DIR__ . '../bin/console', 'do:something', 'param5', 'param6']));
        //...
        //...
        //...
        $this->pool->start();
    }
}

How To Stop The Flow If Anyone Of The Jobs Failed

To fail entire flow if anyone of the jobs failed, you must define fail_on_error as true when you defined the pool.

shepherd:
    pools:
      foo:
        max_processes: 2
        fail_on_error: true        

Then you must handle the error which is thrown by start() method.

<?php
 //...
    
    function performSomething() {
        //...
        try {
            $this->pool->start();    
        } catch (Shepherd\Bundle\ProcessPoolBundle\Exception\ProcessExecutionError $exception) {
            //TODO: Do something...
        }
        //...
    }   
 //...

You can find an example use case in the this blog post: