README

Simple task queue for personal use. Features:

Requires only one database table to work.
Tasks have custom payload and priority.
Failed tasks a retried for up to 10 times.
Queue workers run in a separate process, outside of the web server (fpm).
Task runner is run in yet another separate process, for more fail safety.

Adding tasks

Basically, adding a task is just an insert into the table named taskq. Normally you call the add method of the TaskQueue class, which does that and logging.

Running tasks

Run the bin/taskq script, it will handle the rest. Make sure you restart it when it fails (with cron or supervisord). The logic is:

Grab a file lock. Exit if failed (another instance is running).
Check table taskq, fetch the first task with the highest priority for which the time has come.
Run the task runner process. If it fails, postpone the task. If it succeeds (exit code 0), delete the task.
Go to 2.

Handling tasks

Create a service, register it with yout service controller. For a task named acme.fooBar, there must be a service named acme with a method named fooBar. The method would receive an array of data passed by the caller.

Fail tasks with exceptions.

Database structure

CREATE TABLE IF NOT EXISTS `taskq` (
    `id` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
    `added_at` DATETIME NOT NULL,
    `run_after` DATETIME NOT NULL,
    `priority` INTEGER NOT NULL DEFAULT 0,
    `payload` MEDIUMBLOB NOT NULL,
    `attempts` INTEGER UNSIGNED NOT NULL DEFAULT 0,
    PRIMARY KEY (`id`),
    KEY (`priority`),
    KEY(`run_after`),
    KEY(`attempts`)
) DEFAULT CHARSET utf8;

Changes

2020-07-15:

Fixed autoload issues.

2020-07-01:

Cleaner logging in add().

TODO

Move task runner to a separate class.
Separate lock file per priority.

umonkey / php-taskq

Maintainers

Details