README

A production-ready Laravel package for importing XLSX and CSV files into Eloquent models with column mapping, validation, duplicate handling, relational imports, logging, chunking, and queue support.

Features

Multiple File Formats: Support for CSV, XLSX, and XLS files
Dynamic Column Mapping: Map file columns to model attributes
Relational Imports: Import data with all Laravel Eloquent relation types
Validation: Validate each row using model-defined rules
Duplicate Handling: Skip, update, or fail on duplicate records
Chunking: Process large files in memory-efficient chunks
Queue Support: Process imports asynchronously using Laravel queues
Logging: Track import progress with detailed logs
Events: Listen to import lifecycle events
Field Transformers: Transform data before importing
Preview: Preview file contents before importing
Security: Whitelist allowed models for import

Requirements

PHP 8.1+
Laravel 10.0+

Installation

Install the package via Composer:

composer require elachagui/laravel-smart-importer

Publish the configuration file:

php artisan vendor:publish --tag=smart-importer-config

Run the migrations:

php artisan migrate

Configuration

The configuration file config/smart-importer.php allows you to customize:

return [
    // Models allowed for import (security)
    'allowed_models' => [
        App\Models\User::class,
        App\Models\Product::class,
    ],

    // Default chunk size for processing
    'chunk_size' => 1000,

    // Default duplicate strategy: 'skip', 'update', 'fail'
    'duplicate_strategy' => 'skip',

    // Queue configuration
    'queue' => [
        'enabled' => false,
        'connection' => env('QUEUE_CONNECTION', 'sync'),
        'queue' => 'imports',
    ],

    // Stop on first validation error
    'stop_on_error' => false,

    // Wrap import in database transaction
    'use_transaction' => false,

    // Disable model events during import
    'disable_model_events' => false,

    // File storage configuration
    'storage' => [
        'disk' => 'local',
        'path' => 'imports',
    ],

    // CSV settings
    'csv' => [
        'delimiter' => ',',
        'enclosure' => '"',
        'escape' => '\\',
        'encoding' => 'UTF-8',
    ],

    // Logging
    'logging' => [
        'enabled' => true,
        'channel' => env('LOG_CHANNEL', 'stack'),
    ],
];

Basic Usage

Simple Import

use Elachagui\SmartImporter\Facades\SmartImporter;

$import = SmartImporter::make()
    ->model(App\Models\User::class)
    ->file($request->file('file'))
    ->mapping([
        'Full Name' => 'name',
        'Email Address' => 'email',
        'Phone Number' => 'phone',
    ])
    ->import();

With Duplicate Handling

$import = SmartImporter::make()
    ->model(App\Models\User::class)
    ->file($request->file('file'))
    ->mapping([
        'Name' => 'name',
        'Email' => 'email',
    ])
    ->duplicateStrategy('update') // or 'skip', 'fail'
    ->import();

// Or use convenience methods
->skipDuplicates()
->updateDuplicates()
->failOnDuplicates()

With Field Transformers

use Carbon\Carbon;

$import = SmartImporter::make()
    ->model(App\Models\Product::class)
    ->file($request->file('file'))
    ->mapping([
        'Product Name' => 'name',
        'Price' => 'price',
        'Created Date' => 'created_at',
    ])
    ->transform([
        'price' => fn($value) => floatval(str_replace(['$', ','], '', $value)),
        'created_at' => fn($value) => Carbon::parse($value),
    ])
    ->import();

Queue Processing

$import = SmartImporter::make()
    ->model(App\Models\User::class)
    ->file($request->file('file'))
    ->mapping([
        'Name' => 'name',
        'Email' => 'email',
    ])
    ->queue() // Use default queue
    // or
    ->queue('redis', 'high-priority') // Custom connection and queue
    ->import();

With Options

$import = SmartImporter::make()
    ->model(App\Models\User::class)
    ->file($request->file('file'))
    ->mapping([
        'Name' => 'name',
        'Email' => 'email',
    ])
    ->chunkSize(500)
    ->uniqueBy(['email'])
    ->stopOnError()
    ->withTransaction()
    ->withoutModelEvents()
    ->import();

Preview File

$preview = SmartImporter::make()
    ->file($request->file('file'))
    ->preview(10);

// Returns:
// [
//     'headers' => ['Name', 'Email', 'Phone'],
//     'rows' => [...],
//     'total_rows' => 100,
// ]

Get File Headers

$headers = SmartImporter::make()
    ->file($request->file('file'))
    ->getHeaders();

// Returns: ['Name', 'Email', 'Phone']

Relational Imports

The package supports importing data with all Laravel Eloquent relation types. This allows you to create related records, look up existing records for foreign keys, and sync many-to-many relationships during import.

Supported Relation Types

Relation Type	Description	Processing Phase
`belongsTo`	Lookup/create parent record, set foreign key	Before main model
`hasOne`	Create single child record	After main model
`hasMany`	Create multiple child records	After main model
`belongsToMany`	Sync pivot table (many-to-many)	After main model
`morphOne`	Polymorphic one-to-one	After main model
`morphMany`	Polymorphic one-to-many	After main model
`morphTo`	Inverse polymorphic (set type + id)	Before main model
`morphToMany`	Polymorphic many-to-many	After main model
`morphedByMany`	Inverse polymorphic many-to-many	After main model
`hasOneThrough`	Through intermediate model	After main model
`hasManyThrough`	Through intermediate model	After main model

BelongsTo Relations

Look up an existing parent record (or create it if missing) and automatically set the foreign key.

use App\Models\Product;
use App\Models\Category;

$import = SmartImporter::make()
    ->model(Product::class)
    ->file($file)
    ->mapping([
        'SKU' => 'sku',
        'Name' => 'name',
        'Price' => 'price',
    ])
    ->relations([
        'category' => [
            'type' => 'belongsTo',
            'model' => Category::class,
            'file_column' => 'Category Name',    // Column in CSV/Excel
            'lookup_by' => 'name',               // Column to search in categories table
            'create_if_missing' => true,         // Create category if not found
            'create_mapping' => [                // Optional: map additional fields when creating
                'Category Name' => 'name',
                'Category Slug' => 'slug',
            ],
            'defaults' => [                      // Optional: default values when creating
                'active' => true,
            ],
            'foreign_key' => 'category_id',      // Optional: explicit foreign key (auto-detected)
            'owner_key' => 'id',                 // Optional: parent's key column (default: 'id')
        ],
    ])
    ->import();

Composite Key Lookup:

->relations([
    'warehouse' => [
        'type' => 'belongsTo',
        'model' => Warehouse::class,
        'file_columns' => ['Region Code', 'Warehouse Code'],
        'lookup_columns' => ['region_code', 'code'],
    ],
])

HasOne / HasMany Relations

Create child records after the main model is saved.

use App\Models\Order;
use App\Models\OrderItem;

$import = SmartImporter::make()
    ->model(Order::class)
    ->file($file)
    ->mapping([
        'Order Number' => 'order_number',
        'Total' => 'total',
    ])
    ->relations([
        // HasMany with column mapping
        'items' => [
            'type' => 'hasMany',
            'model' => OrderItem::class,
            'foreign_key' => 'order_id',
            'mapping' => [
                'Item Name' => 'product_name',
                'Quantity' => 'quantity',
                'Unit Price' => 'price',
            ],
        ],
    ])
    ->import();

Multiple Records from Delimited Column:

->relations([
    'variants' => [
        'type' => 'hasMany',
        'model' => ProductVariant::class,
        'foreign_key' => 'product_id',
        'file_column' => 'Sizes',           // e.g., "S,M,L,XL"
        'separator' => ',',
        'mapping' => ['Sizes' => 'size'],
        'defaults' => ['stock' => 0],
    ],
])

Multiple Records from JSON Column:

->relations([
    'items' => [
        'type' => 'hasMany',
        'model' => OrderItem::class,
        'foreign_key' => 'order_id',
        'file_column' => 'Items JSON',
        // File contains: [{"name":"Widget","qty":2},{"name":"Gadget","qty":1}]
    ],
])

BelongsToMany Relations (Many-to-Many)

Sync pivot table relationships from a column containing multiple values.

use App\Models\Product;
use App\Models\Tag;

$import = SmartImporter::make()
    ->model(Product::class)
    ->file($file)
    ->mapping([
        'SKU' => 'sku',
        'Name' => 'name',
    ])
    ->relations([
        'tags' => [
            'type' => 'belongsToMany',
            'model' => Tag::class,
            'file_column' => 'Tags',             // e.g., "electronics,featured,sale"
            'lookup_by' => 'name',               // Find tags by name
            'separator' => ',',                  // How values are separated
            'create_if_missing' => true,         // Create tags that don't exist
            'sync_mode' => 'sync',               // 'sync', 'attach', 'toggle', 'syncWithoutDetaching'
            'pivot_data' => [                    // Optional: data for pivot table
                'assigned_by' => 'import',
            ],
            'pivot_columns' => [                 // Optional: map file columns to pivot columns
                'Tag Priority' => 'priority',
            ],
        ],
    ])
    ->import();

Sync Modes:

Mode	Behavior
`sync`	Replace all existing relations with new ones
`attach`	Add new relations without removing existing
`toggle`	Toggle relations (add if missing, remove if present)
`syncWithoutDetaching`	Add new relations, keep existing ones

Polymorphic Relations

MorphMany - Create polymorphic child records:

use App\Models\Product;
use App\Models\Comment;

$import = SmartImporter::make()
    ->model(Product::class)
    ->file($file)
    ->mapping([
        'SKU' => 'sku',
        'Name' => 'name',
    ])
    ->relations([
        'comments' => [
            'type' => 'morphMany',
            'model' => Comment::class,
            'mapping' => [
                'Review' => 'body',
                'Rating' => 'rating',
            ],
        ],
    ])
    ->import();

MorphTo - Set polymorphic parent:

use App\Models\Comment;
use App\Models\Product;

$import = SmartImporter::make()
    ->model(Comment::class)
    ->file($file)
    ->mapping([
        'Body' => 'body',
    ])
    ->relations([
        'commentable' => [
            'type' => 'morphTo',
            'model' => Product::class,
            'file_column' => 'Product SKU',
            'lookup_by' => 'sku',
            'morph_name' => 'commentable',       // Optional: morph name
            'create_if_missing' => true,
        ],
    ])
    ->import();

MorphToMany - Polymorphic many-to-many:

->relations([
    'tags' => [
        'type' => 'morphToMany',
        'model' => Tag::class,
        'file_column' => 'Tags',
        'lookup_by' => 'name',
        'separator' => ',',
        'create_if_missing' => true,
    ],
])

Through Relations

Create records through an intermediate model.

use App\Models\Country;
use App\Models\User;
use App\Models\Post;

$import = SmartImporter::make()
    ->model(Country::class)
    ->file($file)
    ->mapping([
        'Country Name' => 'name',
    ])
    ->relations([
        'posts' => [
            'type' => 'hasManyThrough',
            'model' => Post::class,
            'through_model' => User::class,
            'first_key' => 'country_id',         // Foreign key on User
            'second_key' => 'user_id',           // Foreign key on Post
            'through_mapping' => [               // Create the intermediate User
                'Author Name' => 'name',
                'Author Email' => 'email',
            ],
            'mapping' => [                       // Create the Post
                'Post Title' => 'title',
                'Post Body' => 'body',
            ],
        ],
    ])
    ->import();

Model-Defined Relations

Define relations directly in your model by implementing the importRelations() method:

use Elachagui\SmartImporter\Contracts\Importable;
use Illuminate\Database\Eloquent\Model;

class Product extends Model implements Importable
{
    public static function importRules(): array
    {
        return [
            'sku' => 'required|string|max:50',
            'name' => 'required|string|max:255',
            'price' => 'required|numeric|min:0',
        ];
    }

    public static function importUniqueBy(): array
    {
        return ['sku'];
    }

    public static function importRelations(): array
    {
        return [
            'category' => [
                'type' => 'belongsTo',
                'model' => Category::class,
                'file_column' => 'category_name',
                'lookup_by' => 'name',
                'create_if_missing' => true,
            ],
            'tags' => [
                'type' => 'belongsToMany',
                'model' => Tag::class,
                'file_column' => 'tags',
                'lookup_by' => 'name',
                'separator' => ',',
                'create_if_missing' => true,
            ],
        ];
    }
}

Then import without specifying relations:

$import = SmartImporter::make()
    ->model(Product::class)
    ->file($file)
    ->mapping([
        'SKU' => 'sku',
        'Name' => 'name',
        'Price' => 'price',
    ])
    ->import(); // Relations are auto-loaded from model

Combining Multiple Relations

$import = SmartImporter::make()
    ->model(Order::class)
    ->file($file)
    ->mapping([
        'Order Number' => 'order_number',
        'Total' => 'total',
    ])
    ->relations([
        // BelongsTo: Set customer_id
        'customer' => [
            'type' => 'belongsTo',
            'model' => Customer::class,
            'file_column' => 'Customer Email',
            'lookup_by' => 'email',
            'create_if_missing' => true,
            'create_mapping' => [
                'Customer Email' => 'email',
                'Customer Name' => 'name',
            ],
        ],
        // HasMany: Create order items
        'items' => [
            'type' => 'hasMany',
            'model' => OrderItem::class,
            'foreign_key' => 'order_id',
            'mapping' => [
                'Product' => 'product_name',
                'Qty' => 'quantity',
                'Price' => 'price',
            ],
        ],
        // BelongsToMany: Sync tags
        'tags' => [
            'type' => 'belongsToMany',
            'model' => Tag::class,
            'file_column' => 'Tags',
            'lookup_by' => 'name',
            'separator' => ',',
            'create_if_missing' => true,
        ],
        // MorphMany: Add comments
        'comments' => [
            'type' => 'morphMany',
            'model' => Comment::class,
            'mapping' => [
                'Note' => 'body',
            ],
        ],
    ])
    ->import();

Relation Configuration Reference

Option	Type	Description
`type`	string	Required. Relation type (belongsTo, hasMany, etc.)
`model`	string	Required. Related model class
`file_column`	string	Single file column to read from
`file_columns`	array	Multiple file columns (for composite keys)
`lookup_by`	string	Column to search by in related table
`lookup_columns`	array	Multiple columns for composite key lookup
`create_if_missing`	bool	Create related record if not found (default: false)
`create_mapping`	array	Column mapping when creating related record
`mapping`	array	Column mapping for hasOne/hasMany child records
`defaults`	array	Default values when creating records
`foreign_key`	string	Explicit foreign key column name
`owner_key`	string	Parent model's key column (default: 'id')
`local_key`	string	Local model's key column (default: 'id')
`separator`	string	Separator for parsing multiple values (default: ',')
`sync_mode`	string	For belongsToMany: sync, attach, toggle (default: 'sync')
`pivot_data`	array	Static data for pivot table
`pivot_columns`	array	Map file columns to pivot columns
`morph_name`	string	Morph name for polymorphic relations
`morph_type_column`	string	Morph type column name
`morph_id_column`	string	Morph ID column name
`through_model`	string	Intermediate model for through relations
`first_key`	string	First key for through relations
`second_key`	string	Second key for through relations
`transformers`	array	Field transformers for this relation

Implementing Importable Interface

For models with validation rules, implement the Importable interface:

use Elachagui\SmartImporter\Contracts\Importable;
use Illuminate\Database\Eloquent\Model;

class User extends Model implements Importable
{
    public static function importRules(): array
    {
        return [
            'name' => 'required|string|max:255',
            'email' => 'required|email|unique:users,email',
            'phone' => 'nullable|string|max:20',
        ];
    }

    public static function importUniqueBy(): array
    {
        return ['email'];
    }

    // Optional: Define relations for import
    public static function importRelations(): array
    {
        return [
            // ... relation definitions
        ];
    }
}

Import Model

The package creates import records to track progress:

use Elachagui\SmartImporter\Models\Import;

$import = Import::find($id);

// Access import data
$import->model;           // Target model class
$import->file_path;       // Path to imported file
$import->total_rows;      // Total rows in file
$import->processed_rows;  // Rows processed
$import->success_rows;    // Successfully imported
$import->failed_rows;     // Failed rows
$import->skipped_rows;    // Skipped (duplicates)
$import->status;          // pending, processing, completed, failed, cancelled
$import->mapping;         // Column mapping (array)
$import->started_at;      // When import started
$import->finished_at;     // When import finished

// Check status
$import->isPending();
$import->isProcessing();
$import->isCompleted();
$import->isFailed();
$import->isCancelled();

// Get progress
$import->getProgressPercentage(); // Returns 0-100

// Get errors
$import->errors; // HasMany relationship to ImportError

Cancelling Imports

$import = SmartImporter::find($id);

SmartImporter::cancel($import);

Events

The package fires several events during the import lifecycle:

use Elachagui\SmartImporter\Events\ImportStarted;
use Elachagui\SmartImporter\Events\ImportCompleted;
use Elachagui\SmartImporter\Events\ImportFailed;
use Elachagui\SmartImporter\Events\RowImported;

// In your EventServiceProvider
protected $listen = [
    ImportStarted::class => [
        SendImportStartedNotification::class,
    ],
    ImportCompleted::class => [
        SendImportCompletedNotification::class,
    ],
    ImportFailed::class => [
        SendImportFailedNotification::class,
    ],
    RowImported::class => [
        ProcessImportedRow::class,
    ],
];

Event Data

// ImportStarted
public Import $import;

// ImportCompleted
public Import $import;
public function getSummary(): array; // Returns summary data

// ImportFailed
public Import $import;
public Throwable $exception;
public function getErrorMessage(): string;
public function getErrorDetails(): array;

// RowImported
public Import $import;
public int $rowNumber;
public array $data;
public string $action; // 'created' or 'updated'
public function wasCreated(): bool;
public function wasUpdated(): bool;

Exception Handling

use Elachagui\SmartImporter\Exceptions\ImportException;

try {
    $import = SmartImporter::make()
        ->model(App\Models\User::class)
        ->file($file)
        ->mapping($mapping)
        ->import();
} catch (ImportException $e) {
    // Handle import-specific errors
    $message = $e->getMessage();
    $rowNumber = $e->getRowNumber();
    $rowData = $e->getRowData();
    $context = $e->getContext();
}

Example Controller

<?php

namespace App\Http\Controllers;

use App\Models\Product;
use App\Models\Category;
use App\Models\Tag;
use Elachagui\SmartImporter\Facades\SmartImporter;
use Elachagui\SmartImporter\Exceptions\ImportException;
use Illuminate\Http\Request;

class ProductImportController extends Controller
{
    public function preview(Request $request)
    {
        $request->validate([
            'file' => 'required|file|mimes:csv,xlsx,xls|max:10240',
        ]);

        $preview = SmartImporter::make()
            ->file($request->file('file'))
            ->preview(10);

        return response()->json($preview);
    }

    public function import(Request $request)
    {
        $request->validate([
            'file' => 'required|file|mimes:csv,xlsx,xls|max:10240',
        ]);

        try {
            $import = SmartImporter::make()
                ->model(Product::class)
                ->file($request->file('file'))
                ->mapping([
                    'SKU' => 'sku',
                    'Product Name' => 'name',
                    'Price' => 'price',
                ])
                ->relations([
                    'category' => [
                        'type' => 'belongsTo',
                        'model' => Category::class,
                        'file_column' => 'Category',
                        'lookup_by' => 'name',
                        'create_if_missing' => true,
                    ],
                    'tags' => [
                        'type' => 'belongsToMany',
                        'model' => Tag::class,
                        'file_column' => 'Tags',
                        'lookup_by' => 'name',
                        'separator' => ',',
                        'create_if_missing' => true,
                    ],
                ])
                ->transform([
                    'price' => fn($v) => floatval(str_replace(['$', ','], '', $v)),
                ])
                ->updateDuplicates()
                ->queue()
                ->import();

            return response()->json([
                'message' => 'Import queued successfully',
                'import_id' => $import->id,
            ]);
        } catch (ImportException $e) {
            return response()->json([
                'message' => $e->getMessage(),
            ], 422);
        }
    }

    public function status(int $id)
    {
        $import = SmartImporter::findWithErrors($id);

        if (!$import) {
            return response()->json(['message' => 'Import not found'], 404);
        }

        return response()->json([
            'status' => $import->status,
            'progress' => $import->getProgressPercentage(),
            'total_rows' => $import->total_rows,
            'processed_rows' => $import->processed_rows,
            'success_rows' => $import->success_rows,
            'failed_rows' => $import->failed_rows,
            'skipped_rows' => $import->skipped_rows,
            'errors' => $import->errors->map(fn($e) => [
                'row' => $e->row_number,
                'errors' => $e->errors,
            ]),
        ]);
    }

    public function cancel(int $id)
    {
        $import = SmartImporter::find($id);

        if (!$import) {
            return response()->json(['message' => 'Import not found'], 404);
        }

        $cancelled = SmartImporter::cancel($import);

        return response()->json([
            'cancelled' => $cancelled,
        ]);
    }
}

Testing

composer test

Security

The package includes security measures:

Model Whitelist: Only models listed in allowed_models config can be imported
File Validation: Only supported file types are processed
Input Validation: All rows are validated before import

Performance Tips

Use queue processing for large files
Adjust chunk_size based on your server's memory
Disable model events with withoutModelEvents() for faster imports
Use batchInsert (default) instead of individual inserts
For imports with hasMany/belongsToMany relations, records are inserted individually (not batched) to support relation processing

License

The MIT License (MIT). Please see License File for more information.

elachagui / laravel-smart-importer

Maintainers

Package info

Statistics

Security

README

Features

Requirements

Installation

Configuration

Basic Usage

Simple Import

With Duplicate Handling

With Field Transformers

Queue Processing

With Options

Preview File

Get File Headers

Relational Imports

Supported Relation Types

BelongsTo Relations

HasOne / HasMany Relations

BelongsToMany Relations (Many-to-Many)

Polymorphic Relations

Through Relations

Model-Defined Relations

Combining Multiple Relations

Relation Configuration Reference

Implementing Importable Interface

Import Model

Cancelling Imports

Events

Event Data

Exception Handling

Example Controller

Testing

Security

Performance Tips

License