README

This lightweight library allows you to perform safer Doctrine migrations during deployment in cluster-based environments like kubernetes where rolling-update takes place. Each migration has two up phases, no down phase.

before
- to be called before any traffic hits the new application version
- typically contains ADD COLUMN etc.
after
- to be called after the deployment is done and no traffic is hitting the old application version
- typically contains DROP COLUMN etc.

You can see Czech talk about this library on YouTube.

Installation:

composer require shipmonk/doctrine-two-phase-migrations

Configuration in Symfony application:

return [
    // ...
    ShipMonk\Doctrine\Migration\Bridge\Symfony\TwoPhaseMigrationsBundle::class => ['all' => true],
];

Then configure it in config/packages/two_phase_migrations.yaml:

two_phase_migrations:
    migrations_dir: '%kernel.project_dir%/migrations'

    # optional parameters:
    # migration_table_name: 'doctrine_migration'
    # migration_class_namespace: 'YourCompany\Migrations'
    # migration_class_prefix: 'Migration'
    # excluded_tables: ['my_tmp_table']
    # template_file_path: '%kernel.project_dir%/migrations/my-template.txt'
    # template_indent: "\t\t"
    # lock_timeout_seconds: 300 # how long migration:run waits for a lock held by a parallel run before failing

The bundle requires symfony/http-kernel ^6.4+. All commands and services are registered automatically, and Doctrine\ORM\EntityManagerInterface is autowired. If you register a custom MigrationExecutor, MigrationAnalyzer, or MigrationVersionProvider service, it will be picked up automatically.

All commands accept an optional Psr\Log\LoggerInterface. When a logger is available (e.g. via MonologBundle), structured log messages with context are emitted. When no logger is injected, commands fall back to Symfony\Component\Console\Logger\ConsoleLogger and write directly to the console output.

Manual service registration (without the bundle)

If your Doctrine\ORM\EntityManagerInterface is autowired, just register few services in your DIC and tag the commands:

_instanceof:
    Symfony\Component\Console\Command\Command:
        tags:
            - console.command

services:
    ShipMonk\Doctrine\Migration\Command\MigrationInitCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationRunCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationSkipCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationCheckCommand:
    ShipMonk\Doctrine\Migration\Command\MigrationGenerateCommand:
    ShipMonk\Doctrine\Migration\MigrationService:
    ShipMonk\Doctrine\Migration\MigrationConfig:
        $migrationsDir: "%kernel.project_dir%/migrations"

    # more optional parameters:
        $migrationClassNamespace: 'YourCompany\Migrations'
        $migrationTableName: 'doctrine_migration'
        $migrationClassPrefix: 'Migration' # will be appended with date('YmDHis') by default
        $excludedTables: ['my_tmp_table'] # migration table ($migrationTableName) is always added to excluded tables automatically
        $templateFilePath: "%kernel.project_dir%/migrations/my-template.txt" # customizable according to your coding style
        $templateIndent: "\t\t" # defaults to spaces
        $lockTimeoutSeconds: 300 # how long migration:run waits for a lock held by a parallel run before failing

Commands:

Initialization:

After installation, you need to create migration table in your database. It is safe to run it even when the table was already initialized. When upgrading from 1.x, running it once also upgrades the existing table to the new schema (it makes finished_at nullable, see Concurrency & execution safety). Only that single column is altered, so it is safe to run against a populated table.

$ bin/console migration:init

# example output:
[info] Initializing migration table doctrine_migration
[info] Migration table doctrine_migration created successfully

Generating new migration:

You can generate migration from database <=> entity diff automatically. This puts all the queries generated by Doctrine to before stage, which will NOT be correct for any destructive actions. Be sure to verify the migration and move the queries to proper stage or adjust them. When no diff is detected, empty migration class is generated.

$ bin/console migration:generate

# example output:
[info] Starting migration generation
[info] 1 schema changes detected
[info] Migration version 20230217063818 generated successfully

The generated file then looks like this:

<?php declare(strict_types = 1);

namespace App\Migrations;

use ShipMonk\Doctrine\Migration\Migration;
use ShipMonk\Doctrine\Migration\MigrationExecutor;

class Migration20230217063818 implements Migration
{

    public function before(MigrationExecutor $executor): void
    {
        $executor->executeQuery('CREATE INDEX IDX_542819F35080ECDE ON my_table (my_column)');
    }

    public function after(MigrationExecutor $executor): void
    {
    }

}

You can adjust it by providing custom $templateFilePath to MigrationConfig, but it needs to implement Migration interface.

Status verification:

You can check awaiting migrations and entity sync status:

$ bin/console migration:check

# example success output:
[info] Starting migration check
[info] Phase before fully executed, no awaiting migrations
[info] Phase after fully executed, no awaiting migrations
[info] Database is synced with entities, no migration needed
[info] Migration check completed

$ bin/console migration:check

# example failure output:
[info] Starting migration check
[info] Phase before fully executed, no awaiting migrations
[error] Phase after has executed migrations not present in /app/migrations: 20220208123456
[warning] Database is not synced with entities, 1 missing updates
[info] Migration check completed

Skipping all migrations:

You can also mark all migrations as already executed, e.g. when you just created fresh schema from entities. This will mark all not executed migrations in all stages as migrated.

$ bin/console migration:skip

# example output:
[info] Starting migration skip
[info] Found 1 migrations to skip in phase after
[info] Migration 20230214154154 phase after skipped
[info] Migration skip completed, 1 skipped

Executing migration:

Execution is performed without any interaction and does not fail when no migration is present for execution.

$ bin/console migration:run before

# example output:
[info] Starting migration execution (phase before)
[info] 1 pending migrations found
[info] Executing migration 20220224045126 phase before
[info] Migration 20220224045126 phase before executed successfully, 0.032 s elapsed
[info] Migration execution completed (phase before)

$ bin/console migration:run after

# example output:
[info] Starting migration execution (phase after)
[info] 1 pending migrations found
[info] Executing migration 20220224045126 phase after
[info] Migration 20220224045126 phase after executed successfully, 0.033 s elapsed
[info] Migration execution completed (phase after)

When executing all the migrations (e.g. in test environment) you probably want to achieve one-by-one execution. You can do that by:

$ bin/console migration:run both

# example output:
[info] Starting migration execution (phase both)
[info] 2 pending migrations found
[info] Executing migration 20220224045126 phase before
[info] Migration 20220224045126 phase before executed successfully, 0.032 s elapsed
[info] Executing migration 20220224045126 phase after
[info] Migration 20220224045126 phase after executed successfully, 0.033 s elapsed
[info] Migration execution completed (phase both)

Concurrency & execution safety

migration:run is designed to be safe to run multiple times and in parallel (e.g. during a rolling deployment where several instances may start the command at once). "Safe" here means it will never silently corrupt your database — not that every invocation finishes the work. Two mechanisms provide this:

Parallel runs are serialized by a database lock

Before doing anything, migration:run acquires a database-level lock, so only one run executes migrations at a time:

MySQL / MariaDB – a named lock via GET_LOCK() / RELEASE_LOCK()
PostgreSQL – a session-level advisory lock via pg_try_advisory_lock() / pg_advisory_unlock()
other platforms (incl. SQLite) – no-op (SQLite already serializes writers at the filesystem level)

A parallel run waits up to lock_timeout_seconds (default 300) for the lock and then proceeds once the holder finishes. If the timeout is exceeded, it aborts with exit code 2 instead of running concurrently. The lock is bound to the connection session, so it is released automatically if a runner is killed.

Interrupted migrations block all further runs until resolved

Each migration is recorded in two steps: a row with started_at (and finished_at = NULL) is committed before the migration body runs, and finished_at is set after it succeeds. If a runner is killed mid-migration, a finished_at = NULL row is left behind. This start marker is intentionally written outside any transaction, so it relies on the connection committing it immediately (autocommit, the Doctrine default); the detection guarantee does not hold for connections configured with auto_commit: false.

Because such a migration may be only partially applied, every subsequent migration:run fails loudly (exit code 1) instead of re-executing potentially non-idempotent SQL:

$ bin/console migration:run before

# example output:
[info] Starting migration execution (phase before)
[error] Migration execution aborted, found 1 unfinished migration(s) from a previously interrupted run, manual resolution is required

To recover, inspect the database to determine what the interrupted migration actually applied, then resolve the finished_at = NULL row in the migration table manually:

delete the row to re-run the migration from scratch (only safe if nothing was applied or the migration is idempotent), or
set its finished_at to mark it as completed (if you verified everything was applied or finished it by hand).

migration:run exit codes: 0 success, 1 unfinished migration found (manual resolution required), 2 lock could not be acquired, 3 migration table missing or not upgraded (run migration:init). migration:check also reports unfinished migrations (and adds 8 to its exit code bitmask).

Advanced usage

Run custom code for each executed query:

You can hook into migration execution by implementing MigrationExecutor interface and registering your implementations as a service. Implement executeQuery() to run checks or other code before/after each query. Interface of this method mimics interface of Doctrine\DBAL\Connection::executeQuery().

Alter generated migration SQLs:

You can implement custom MigrationAnalyzer and register it as a service. This allows you to alter generated SQLs (e.g. add ALGORITHM=INSTANT) and assign them to proper phase.

Hook to migration execution:

If you pass Psr\EventDispatcher\EventDispatcherInterface to MigrationService, you can hook into migration execution. Dispatched events are:

MigrationExecutionStartedEvent
MigrationExecutionSucceededEvent
MigrationExecutionFailedEvent

All events have MigrationPhase enum and Migration instance available.

Run all queries within transaction:

You can change your template (or a single migration) to extend TransactionalMigration. That causes each phases to be executed within migration. Be aware that many databases (like MySQL) does not support transaction over DDL operations (ALTER and such).

Checking execution duration:

Migration table has started_at and finished_at columns with datetime data with microseconds. But those columns are declared as VARCHARs by default, because there is no microsecond support in doctrine/dbal yet. That may complicate datetime manipulations (like duration calculation). You can adjust the structure to your needs (e.g. use DATETIME(6) for MySQL) manually in some migration.

finished_at is nullable: a NULL value marks a migration that was started but never finished (see Concurrency & execution safety).

+----------------+--------+-----------------------------+---------------------------+
| version        | phase  | started_at                  | finished_at               |
+----------------+--------+-----------------------------+---------------------------+
| 20220224045126 | before | 2023-02-17 05:19:50.225048 | 2023-02-17 05:19:50.672871 |
| 20220224045126 | after  | 2023-02-17 05:23:11.265727 | 2023-02-17 05:23:11.982982 |
+------------------------------------------------------+----------------------------+

Differences from doctrine/migrations

The underlying diff checking and generation is equal to what happens in doctrine/migrations as it uses doctrine/dbal features. Main difference is that we do not provide any downgrade phases.

This library is aiming to provide only core functionality needed for safe migrations within rolling-update deployments. Basically all the logic is inside MigrationService, which has only ~300 lines. We try to keep it as lightweight as possible, we do not plan to copy features from doctrine/migrations.

shipmonk / doctrine-two-phase-migrations

Maintainers

Package info

Statistics

Security