README

A queue-based data migration system for Laravel that separates structural database changes from data-heavy operations. Dispatch long-running data transformations as queued jobs with priority scheduling, type-safety, and production safeguards.

Why?

Standard Laravel migrations run synchronously during deployment. If you need to update 10 million rows, your deployment script will time out. This package dispatches data migrations as queued jobs so your deployment finishes immediately while data churns safely in the background.

Requirements

PHP 8.3+
Laravel 12 or 13

Installation

composer require NorthBees/data-migrations

The service provider is auto-discovered. Publish the config and run migrations:

php artisan vendor:publish --tag=data-migrations-config
php artisan migrate

The package automatically loads its own migrations, so php artisan migrate is all you need. If you'd prefer to have the migration files in your application (e.g. to customise the schema), you can publish them:

php artisan vendor:publish --tag=data-migrations-migrations

This copies the migration files into your application's database/migrations/ directory. Once published, the package will no longer auto-load its copies — your published versions take precedence.

Configuration

// config/data-migrations.php

return [
    'table'        => 'data_migrations',
    'failed_table' => 'failed_data_migrations',
    'path'         => database_path('data-migrations'),

    // Low-priority jobs are delayed until this window
    'off_peak_start' => '00:00',
    'off_peak_end'   => '06:00',

    // Queue names
    'queue_high' => 'default',
    'queue_low'  => 'data-migrations-low',

    // Safety
    'prevent_deletes_in_production' => false,
];

Usage

Creating a data migration

php artisan make:data-migration fix_users_email --priority=high --type=alter

This generates a timestamped file in database/data-migrations/:

<?php

declare(strict_types=1);

namespace Database\DataMigrations;

use NorthBees\DataMigrations\Contracts\DataMigrationInterface;
use NorthBees\DataMigrations\Enums\MigrationPriority;
use NorthBees\DataMigrations\Enums\MigrationType;

class FixUsersEmail implements DataMigrationInterface
{
    public function up(): void
    {
        // Perform data migration logic here.
    }

    public function getPriority(): MigrationPriority
    {
        return MigrationPriority::HIGH;
    }

    public function getType(): MigrationType
    {
        return MigrationType::ALTER;
    }

    public function isTransactional(): bool
    {
        return true;
    }
}

Options:

Option	Values	Default
`--priority`	`high`, `low`	`high`
`--type`	`insert`, `alter`, `delete`	`insert`

Running data migrations

php artisan data-migrate:run

This will:

Scan database/data-migrations/ for migration files.
Filter out any already completed (recorded in data_migrations with a non-null completed_at).
Assign a batch number and record each pending migration.
Dispatch a ProcessDataMigration job for each one.

Checking status

php artisan data-migrate:status

Outputs a table showing each migration's batch, type, priority, and status (PENDING, COMPLETED, or FAILED).

Retrying failed migrations

# Retry a specific failed migration by ID
php artisan data-migrate:retry 5

# Retry all failed migrations
php artisan data-migrate:retry --all

How it works

Priority routing

High priority jobs are dispatched to the queue_high queue (default: default) immediately.
Low priority jobs are dispatched to the queue_low queue (default: data-migrations-low) and delayed until the configured off-peak window if the current time is outside it.

Atomicity

When isTransactional() returns true on a migration, its up() method runs inside a database transaction. If it throws, the transaction is rolled back and nothing is written to completed_at. Set it to false for migrations that handle their own chunking or don't need atomic execution.

Error handling

If a job fails:

The data_migrations record remains without a completed_at timestamp.
A record is written to failed_data_migrations with the full exception message and trace.
The migration can be retried with data-migrate:retry.

Delete guard

Set prevent_deletes_in_production to true in the config to block any migration with type delete from executing in the production environment. The job will throw a RuntimeException with a descriptive message.

Database schema

`data_migrations`

Column	Type	Description
`id`	bigint (PK)	Unique identifier
`migration`	string	Filename/class identifier
`batch`	integer	Groups migrations from the same run
`type`	string	`insert`, `alter`, or `delete`
`priority`	string	`high` or `low`
`completed_at`	timestamp (nullable)	Set on successful completion

`failed_data_migrations`

Column	Type	Description
`id`	bigint (PK)	Unique identifier
`migration`	string	Filename/class identifier
`batch`	integer	The batch it was attempted in
`type`	string	`insert`, `alter`, or `delete`
`priority`	string	`high` or `low`
`exception`	text	Exception message and stack trace
`failed_at`	timestamp	When the failure occurred

Deploying

To automatically run data migrations as part of your deployment, add a post-deploy script to your application's composer.json:

{
    "scripts": {
        "post-deploy": [
            "@php artisan migrate --force",
            "@php artisan data-migrate:run"
        ]
    }
}

Then call it from your deployment pipeline:

composer post-deploy

Or invoke the command directly in your CI/CD script (e.g. GitHub Actions, Forge deploy script, Envoyer):

php artisan migrate --force
php artisan data-migrate:run

The key point is that data-migrate:run returns immediately after dispatching jobs to the queue. Your deployment is not blocked by long-running data operations — those execute in the background via your queue workers.

Testing

composer test

Or directly:

vendor/bin/pest

Static analysis

vendor/bin/phpstan analyse

Code style

vendor/bin/pint

License

MIT

northbees / data-migrations

Maintainers

Package info

Statistics

Security