discoverygarden/dgi_migrate

v3.20.0 2026-02-23 18:01 UTC

Requires

• discoverygarden/dgi_saxon_helper: ^1
• discoverygarden/foxml: ^1
• discoverygarden/islandora_drush_utils: ^1 || ^2
• discoverygarden/update-helper: ^1
• drupal/controlled_access_terms: ^2
• drupal/imagemagick: ^3.2
• drupal/islandora: ^2
• drupal/migrate_directory: ^1 || ^2
• drupal/migrate_plus: ^5.1 || ^6
• drupal/migrate_tools: ^6.1.0
• drupal/paragraphs: ^1
• drupal/pathauto: ^1.8
• iqb/substream: dev-master
• symfony/mime: ^5 || ^6

Requires (Dev)

• drupal/devel: ^4

Suggests

• discoverygarden/entity_reference_integrity_extra: Deal with addtional 'entity_reference' fields.
• drupal/entity_reference_integrity: Used to check things when a migration elects to 'manage_orphans'.

GPL-3.0-or-later

README

Introduction

A module to facilitate I7 to Modern Islandora migration.

Table of Contents

• Features
• Included Modules
• Requirements
• Installation
• Usage
• Resources
• Troubleshooting/Issues
• Maintainers and Sponsors
• Development/Contribution
• License

Features

An improved migration import command migrate:batch-import is included. An example FOXML migration that can be used as a starting point is provided. It illustrates the usage of migrate plugins that have been created to facilitate FOXML and large data processing.

• dgi_migrate.process.xml.xpath is limited to xpath 1.0.

Some migration helper scripts are also included. Refer to the readme for more details.

Included modules

DGI Migrate has a suite of submodules to assist in the migration process.

• devel
• dgi_migrate_big_set_overrides
• dgi_migrate_dspace
• dgi_migrate_edtf_validator
• dgi_migrate_foxml_standard_mods
• dgi_migrate_imagemagick_cleanup
• dgi_migrate_paragraphs
• dgi_migrate_regenerate_pathauto_aliases

Requirements

This module requires the following modules/libraries:

• migrate
• migrate_plus
• migrate_directory

  NOTE: For better performance and memory optimisation, it is advised to use the module with this patch

• islandora
• islandora_drush_utils
• foxml

Installation

Install as usual, see this for further information.

Usage

migrate:batch-import beer_node_revision --idlist=1:2,2:3,3:5 --user=islandora

dgi-migrate:rollback beer_user --idlist=5 --user=islandora

Configuration

Search API, direct/immediate indexing

The suppression search_api's "immediate indexing"/index_directly functionality (which can cause instability in long processes, when search_api attempts to index potentially large sets of entities at the end of the request) is enabled by default, but can be configured by multiple means.

• The DGI_MIGRATE_SUPPRESS_DIRECT_INDEXING_DURING_MIGRATIONS environment variables takes precedence if set. The string true should enable (case-sensitive!); while any other non-empty value should disable.
• In config: dgi_migrate.settings:suppress_direct_indexing_during_migrations, as a boolean flag.

Entity update process

Migrations can update existing entities (even without the --update flag), should the process identify an existing entity; however, during this process, if all the source properties are not provided for the destination rows that are mapped, it might end up erasing the related fields/properties from the entities in the database.

To permit partial sources to be provided, we have implemented a process to try to track which destination properties did not have a corresponding source property, by implementing some handling around the get plugin. Where there is no source properties for an existing entity, existing values should be left intact. If a source property is provided with an empty value, the existing value should be erased.

This functionality is enabled by default; however:

• it only affects migrations making use of our dgi_revisioned_entity destination plugin
• it might be disabled by specifying an environment variable DGI_MIGRATE_TRACKING_GET_DISABLED=true.

NOTE: There may be other plugins that access properties from the row by means other than using the get plugin, to which the current implementation is blind, for example, dgi_migrate.process.entity_query, via its conditions key.

Troubleshooting/Issues

Having problems or solved a problem? Contact discoverygarden.

Known Issues:

• php://filter use can lead to large memory usage
  • we should probably look at rolling another stream wrapper to wrap up our usage of OpenSSL to Base64 decode
• There are some expensive assertions made in the code, particularly regarding binary datastream content with digests. Assertions should typically be disabled in production environments, so these shouldn't have any impact on execution there; however, in development environments, could potentially lead to issues, especially with larger datastreams, exacerbated by the php://filter usage to Base64-decode the contents
  • hesitant to remove the assertions without having any other mechanism to
  • could instead roll some unit tests?
• For archival FOXML, a patch for iqb/substream may be necessary to address a bug in the stream wrapper implementation; otherwise, there may be MigrateExceptions thrown which point back at a failure to write to the filtering process, while the actual issue pops out from the stream_copy_to_stream() proper.

Resources

DGI migrate enhances Drupal's core migration framework and provides plugins to facilitate migration of legacy Islandora data to Modern Islandora. For more details regarding Drupal's migrate API and workflow, refer to these documents:

• Migrate API Overview
• Drush Migrate Commands
• Drupal Migration Tutorial

Maintainers and Sponsors

Current maintainers:

• discoverygarden

Sponsors:

• FLVC

Development/Contribution

If you would like to contribute to this module, please check out github's helpful Contributing to projects documentation and Islandora community's Documention for developers to create an issue or pull request and/or contact discoverygarden.

License

GPLv3