spatie/laravel-long-running-tasks

Handle long running tasks in a Laravel app

0.0.4 2024-09-15 19:49 UTC

Requires

• php: ^8.2
• illuminate/contracts: ^10.0|^11.0
• spatie/laravel-package-tools: ^1.14.0
• spatie/test-time: ^1.3

Requires (Dev)

• laravel/pint: ^1.0
• nunomaduro/collision: ^7.0|^8.1
• orchestra/testbench: ^8.8|^9.0
• pestphp/pest: ^2.20
• pestphp/pest-plugin-arch: ^2.5
• pestphp/pest-plugin-laravel: ^2.0
• spatie/laravel-ray: ^1.26

MIT

• Freek Van der Herten

• laravel
• laravel-long-running-tasks

README

Some services, like AWS Rekognition, allow you to start a task on their side. Instead of sending a webhook when the task is finished, the services expects you to regularly poll to know when it is finished (or get an updated status).

This package can help you monitor such long-running tasks that are executed externally.

You do so by creating a task like this.

use Spatie\LongRunningTasks\LongRunningTask;
use Spatie\LongRunningTasks\Enums\TaskResult;
use Spatie\LongRunningTasks\LongRunningTask;

class MyTask extends LongRunningTask
{
    public function check(LongRunningTaskLogItem $logItem): TaskResult
    {
        // get some information about this task
        $meta = $logItem->meta
    
        // do some work here
        $allWorkIsDone = /* ... */
       
        // return wheter we should continue the task in a new run
        
         return $allWorkIsDone
            ? TaskResult::StopChecking
            : TaskResult::ContinueChecking
    }
}

When TaskResult::ContinueChecking is returned, this check function will be called again in 10 seconds (as defined in the default_check_frequency_in_seconds of the config file).

After you have created your task, you can start it like this.

MyTask::make()->meta($anArray)->start();

The check method of MyTask will be called every 10 seconds until it returns TaskResult::StopChecking

Installation

You can install the package via composer:

composer require spatie/laravel-long-running-tasks

You can publish and run the migrations with:

php artisan vendor:publish --tag="long-running-tasks-migrations"
php artisan migrate

You can publish the config file with:

php artisan vendor:publish --tag="long-running-tasks-config"

This is the contents of the published config file:

return [
    /*
     * Behind the scenes, this packages use a queue to call tasks.
     * Here you can choose the queue that should be used by default.
     */
    'queue' => 'default',

    /*
     * If a task determines that it should be continued, it will
     * be called again after this amount of time
     */
    'default_check_frequency_in_seconds' => 10,

    /*
     * The default class that implements a strategy for determining the check frequency in seconds.
     * - `DefaultCheckStrategy` will check the task every `LongRunningTaskLogItem::check_frequency_in_seconds` seconds.
     * - `StandardBackoffCheckStrategy` will check the task every `LongRunningTaskLogItem::check_frequency_in_seconds` seconds,
     *    but will increase the frequency with the multipliers 1, 6, 12, 30, 60, with the maximum being 60 times the original frequency.
     *    With the default check frequency, this translates to 10, 60, 120, 300, and 600 seconds between checks.
     * - `LinearBackoffCheckStrategy` will check the task every `LongRunningTaskLogItem::check_frequency_in_seconds` seconds,
     *   but will increase the frequency linearly with each attempt, up to a maximum multiple of 6 times the original frequency.
     * - `ExponentialBackoffCheckStrategy` will check the task every `LongRunningTaskLogItem::check_frequency_in_seconds` seconds,
     *   but will increase the frequency exponentially with each attempt, up to a maximum of 6 times the original frequency.
     */
    'default_check_strategy_class' => Spatie\LongRunningTasks\Strategies\DefaultCheckStrategy::class,

    /*
     * When a task is not completed in this amount of time,
     * it will not run again, and marked as `didNotComplete`.
     */
    'keep_checking_for_in_seconds' => 60 * 5,

    /*
     * The model that will be used by default to track
     * the status of all tasks.
     */
    'log_model' => Spatie\LongRunningTasks\Models\LongRunningTaskLogItem::class,

    /*
     * The job responsible for calling tasks.
     */
    'task_job' => Spatie\LongRunningTasks\Jobs\RunLongRunningTaskJob::class,
];

This package makes use of queues to call tasks again after a certain amount of time. Make sure you've set up queues in your Laravel app.

Usage

To monitor a long-running task on an external service, you should define a task class. It should extend the Spatie\LongRunningTasks\LongRunningTask provided by the package.

It's check function should perform the work you need it to do and return a TaskResult. When returning TaskResult::StopChecking the task will not be called again. When returning TaskResult::ContinueChecking it will be called again in 10 seconds by default.

use Spatie\LongRunningTasks\LongRunningTask;
use Spatie\LongRunningTasks\Enums\TaskResult;

class MyTask extends LongRunningTask
{
    public function check(LongRunningTaskLogItem $logItem): TaskResult
    {
        // get some information about this task
        $meta = $logItem->meta // returns an array
    
        // do some work here
        $allWorkIsDone = /* ... */
       
        // return wheter we should continue the task in a new run
        
         return $allWorkIsDone
            ? TaskResult::StopChecking
            : TaskResult::ContinueChecking
    }
}

To start the task above, you can call the start method.

MyTask::make()->start();

This will create a record in the long_running_task_log_items table that tracks the progress of this task. The check method of MyTask will be called every 10 seconds until it returns TaskResult::StopChecking.

Adding meta data

In most cases, you'll want to give a task some specific data it can act upon. This can be done by passing an array to the meta method.

MyTask::make()->meta($arrayWithMetaData)->start();

Alternatively, you could also pass it to the start method.

MyTask::make()->start($arrayWithMetaData);

The given array will be available on the LongRunningTaskLogItem instance that is passed to the check method of your task.

class MyTask extends LongRunningTask
{
    public function check(LongRunningTaskLogItem $logItem): TaskResult
    {
        // get some information about this task
        $meta = $logItem->meta // returns an array

        // rest of method
    }
}

Customizing the check interval

By default, when the check method of your task returns TaskResult::ContinueChecking, it will be called again in 10 seconds. You can customize that timespan by changing the value of the default_check_frequency_in_seconds key in the long-running-tasks config file.

You can also specify a check interval on your task itself.

class MyTask extends LongRunningTask
{
    public int $checkFrequencyInSeconds = 20;
}

To specify a checking interval on a specific instance of a task, you can use the checkInterval method.

MyTask::make()
   ->checkFrequencyInSeconds(30)
   ->start();

Customizing the check strategy

By default, the package uses the DefaultCheckStrategy to determine the check frequency in seconds, which always returns the value of the check_frequency_in_seconds attribute.
You can customize the strategy that should be used by changing the value of the default_check_strategy_class key in the long-running-tasks config file.

You can also specify a strategy on your task itself.

use Spatie\LongRunningTasks\Strategies\StandardBackoffCheckStrategy;

class MyTask extends LongRunningTask
{
    public string $checkStrategy = StandardBackoffCheckStrategy::class;
}

To specify a check strategy on a specific instance of a task, you can use the checkStrategy method.

MyTask::make()
   ->checkStrategy(StandardBackoffCheckStrategy::class)
   ->start();

Using a different queue

This package uses queues to call tasks again after a certain amount of time. By default, it will use the default queue. You can customize the queue that should be used by changing the value of the queue key in the long-running-tasks config file.

You can also specify a queue on your task itself.

class MyTask extends LongRunningTask
{
    public string $queue = 'my-custom-queue';
}

To specify a queue on a specific instance of a task, you can use the onQueue method.

MyTask::make()
   ->queue('my-custom-queue')
   ->start();

Tracking the status of tasks

For each task that is started, a record will be created in the long_running_task_log_items table. This record will track the status of the task.

The LongRunningTaskLogItem model has a status attribute that can have the following values:

• pending: The task has not been started yet.
• running: The task is currently running.
• completed: The task has completed.
• failed: The task has failed. Probably an unhanded exception was thrown.
• didNotComplete: The task did not complete in the given amount of time.

The table also contains these properties:

• task: The fully qualified class name of the task.
• queue: The queue the task is running on.
• check_frequency_in_seconds: The amount of time in seconds that should pass before the task is checked again.
• meta: An array of meta data that was passed to the task.
• last_check_started_at: The date and time the task was started.
• last_check_ended_at: The date and time the task was ended.
• stop_checking_at: The date and time the task should stop being checked.
• lastest_exception: An array with keys message and trace that contains the latest exception that was thrown.
• run_count: The amount of times the task has been run.
• attempt: The amount of times the task has been attempted after a failure occurred.
• created_at: The date and time the record was created.

Preventing never-ending tasks

The package has a way of preventing task to run indefinitely.

When a task is not completed in the amount of time specified in the keep_checking_for_in_seconds key of the long-running-tasks config file, it will not run again, and marked as didNotComplete.

You can customize that timespan on a specific task.

class MyTask extends LongRunningTask
{
    public int $keepCheckingForInSeconds = 60 * 10;
}

You can also specify the timespan on a specific instance of a task.

MyTask::make()
   ->keepCheckingForInSeconds(60 * 10)
   ->start();

Handling exceptions

When an exception is thrown in the check method of your task, it will be caught and stored in the latest_exception attribute of the LongRunningTaskLogItem model.

Optionally, you can define an onFailure method on your task. This method will be called when an exception is thrown in the check method.

use Spatie\LongRunningTasks\LongRunningTask;
use Spatie\LongRunningTasks\Enums\TaskResult;

class MyTask extends LongRunningTask
{
    public function check(LongRunningTaskLogItem $logItem): TaskResult
    {
        throw new Exception('Something went wrong');
    }

    public function onFail(LongRunningTaskLogItem $logItem, Exception $exception): ?TaskResult
    {
        // handle the exception
    }
}

You can let the onFail method return a TaskResult. When it returns TaskResult::ContinueChecking, the task will be called again. If it doesn't return anything, the task will not be called again.

Events

Using your own model

If you need extra fields or functionality on the LongRunningTaskLogItem model, you can create your own model that extends the LongRunningTaskLogItem model provided by this package.

namespace App\Models;

use Spatie\LongRunningTasks\Models\LongRunningTaskLogItem as BaseLongRunningTaskLogItem;

class LongRunningTaskLogItem extends BaseLongRunningTaskLogItem
{
    // your custom functionality
}

You should then update the log_model key in the long-running-tasks config file to point to your custom model.

// in config/long-running-tasks.php

return [
    // ...

    'log_model' => App\Models\LongRunningTaskLogItem::class,
];

To fill the extra custom fields of your model, you could use the creating and updating events. You could use the meta property to pass data to the model.

namespace App\Models;

use Spatie\LongRunningTasks\Models\LongRunningTaskLogItem as BaseLongRunningTaskLogItem;

class LongRunningTaskLogItem extends BaseLongRunningTaskLogItem
{
    protected static function booted()
    {
        static::creating(function ($logItem) {
            $customValue = $logItem->meta['some_key'];
            
            // optionally, you could unset the custom value from the meta array
            unset($logItem->meta['some_key']);
        
            $logItem->custom_field = $customValue;
        });
    }
}

Using your own job

By default, the package uses the RunLongRunningTaskJob job to call tasks. If you want to use your own job, you can create a job that extends the RunLongRunningTaskJob job provided by this package.

namespace App\Jobs;

use Spatie\LongRunningTasks\Jobs\RunLongRunningTaskJob as BaseRunLongRunningTaskJob;

class RunLongRunningTaskJob extends BaseRunLongRunningTaskJob
{
    // your custom functionality
}

You should then update the task_job key in the long-running-tasks config file to point to your custom job.

// in config/long-running-tasks.php

return [
    // ...

    'task_job' => App\Jobs\RunLongRunningTaskJob::class,
];

Events

The package fires events that you can listen to in your application to perform additional actions when certain events occur.

All of these events have a property $longRunningTaskLogItem that contains a LongRunningTaskLogItem model.

Spatie\LongRunningTasks\Events\TaskRunStarting

This event will be fired when a task is about to be run.

Spatie\LongRunningTasks\Events\TaskRunEnded

This event will be fired when a task has ended.

Spatie\LongRunningTasks\Events\TaskCompleted

This event will be fired when a task has completed.

Spatie\LongRunningTasks\Events\TaskRunFailed

This event will be fired when a task has failed.

Spatie\LongRunningTasks\Events\TaskRunDidNotComplete

This event will be fired when a task did not complete in the given amount of time.

Spatie\LongRunningTasks\Events\DispatchingNewRunEvent

This event will be fired when a new run of a task is about to be dispatched.

Testing

composer test

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Credits

• Freek Van der Herten
• All Contributors

License

The MIT License (MIT). Please see License File for more information.