pixielity/laravel-import-export

Import/Export module with auto-discovery and queue processing

v1.0.2 2026-02-09 14:07 UTC

Requires

• php: ^8.5
• illuminate/database: ^12.0
• illuminate/queue: ^12.0
• pixielity/laravel-foundation: *
• pixielity/laravel-framework: *
• pixielity/laravel-support: *

Requires (Dev)

• mockery/mockery: ^1.6
• orchestra/testbench: ^10.0
• phpunit/phpunit: ^11.0

MIT

• Pixielity Team

• queue
• excel
• csv
• import
• export
• laravel
• batch-processing

README

Import/Export Module

Complete import/export system with queue processing, real-time progress tracking, and auto-discovery.

Features

✅ Auto-Discovery - Models implementing Importable/Exportable are automatically discovered ✅ Queue Processing - Background processing with progress tracking ✅ Real-time Updates - Pusher broadcasting for live progress ✅ Email Notifications - Notifications when import/export completes (with file attachment for exports) ✅ Error Tracking - Detailed logs for failed rows ✅ File Management - Auto-cleanup after 7 days ✅ Template Generation - Download Excel templates ✅ Sample Data - Manage sample data files per module ✅ Type Safety - Enums for status values and DTOs for API responses ✅ Validation - Request DTOs with automatic validation

Quick Start

1. Run Migrations

php artisan migrate

2. Add to Model

use Modules\ImportExport\Contracts\Importable;
use Modules\ImportExport\Contracts\Exportable;
use Modules\ImportExport\Traits\HasImport;
use Modules\ImportExport\Traits\HasExport;

class Facility extends Model implements Importable, Exportable
{
    use HasImport, HasExport;

    // Define column mapping (single source of truth)
    public function getImportExportMap(): array
    {
        return [
            'Name' => 'name',
            'Code' => 'code',
            'City' => 'city.name',  // Supports relationships
            'Latitude' => 'latitude',
            'Longitude' => 'longitude',
        ];
    }

    // Import validation rules
    public function getImportRules(): array
    {
        return [
            'Name' => 'required|string|max:255',
            'Code' => 'required|string|unique:facilities,code',
            'Latitude' => 'nullable|numeric|between:-90,90',
            'Longitude' => 'nullable|numeric|between:-180,180',
        ];
    }

    // Optional: Additional export columns
    public function getExportAdditionalColumns(): array
    {
        return [
            'Total Visits' => fn($model) => $model->visits()->count(),
            'Created By' => fn($model) => $model->createdBy->name ?? 'N/A',
        ];
    }
}

3. Configure Queue

# .env
QUEUE_CONNECTION=redis
IMPORT_EXPORT_QUEUE_NAME=imports

4. Start Queue Worker

php artisan queue:work --queue=imports

5. Use API

# List importable models
GET /api/v1/import

# Download template
GET /api/v1/import/facilities/template

# Upload file
POST /api/v1/import/facilities
Body: file=facilities.xlsx

# Check status
GET /api/v1/import/status/123

# Export data
GET /api/v1/export/facilities?status=active

# Download export
GET /api/v1/export/download/456

API Endpoints

Import

• GET /api/v1/import - List importable models
• POST /api/v1/import/{identifier} - Upload file
• GET /api/v1/import/status/{batchId} - Check status
• GET /api/v1/import/logs/{batchId} - Get error logs
• GET /api/v1/import/{identifier}/template - Download template

Export

• GET /api/v1/export - List exportable models
• GET /api/v1/export/{identifier} - Request export
• GET /api/v1/export/status/{batchId} - Check status
• GET /api/v1/export/download/{batchId} - Download file

Sample Data

• GET /api/v1/sample-data - List sample files
• GET /api/v1/sample-data/{identifier} - Download sample

Real-time Updates

// Subscribe to import progress
const channel = pusher.subscribe(`import.${batchId}`);

channel.bind("ImportStarted", (data) => {
  console.log("Import started");
});

channel.bind("ImportProgressUpdated", (data) => {
  updateProgressBar(data.progress);
});

channel.bind("ImportCompleted", (data) => {
  showSuccess(`${data.successful_rows} rows imported`);
});

Configuration

Publish configuration:

php artisan vendor:publish --tag=import-export-config

Edit config/import-export.php:

return [
    'upload' => [
        'max_size' => 10240, // KB
    ],
    'retention' => [
        'import_files' => 7, // Days
        'export_files' => 7, // Days
    ],
];

Sample Data

Place sample files in module's Assets/Data folder:

modules/Facilities/Assets/Data/
├── facilities.xlsx
└── facilities.csv

Advanced Usage

Using DTOs in Your Code

use Modules\ImportExport\Data\ImportResultData;
use Modules\ImportExport\Data\ExportResultData;
use Modules\ImportExport\Enums\BatchStatus;

// Check import status
$result = $importService->getStatus($batchId);
// Returns: ImportResultData

if ($result->isCompleted()) {
    echo "Success rate: {$result->getSuccessRate()}%";
}

// Check export status
$result = $exportService->getStatus($batchId);
// Returns: ExportResultData

if ($result->isReadyForDownload()) {
    echo "Download: {$result->download_url}";
}

// Using enums
if ($batch->status === BatchStatus::PROCESSING) {
    echo "Progress: {$batch->getProgressPercentage()}%";
}

Custom Transformations

public function transformImportRow(array $row): array
{
    // Convert city name to ID
    if (isset($row['City'])) {
        $city = City::where('name', $row['City'])->first();
        $row['city_id'] = $city?->id;
        unset($row['City']);
    }

    return $row;
}

Additional Export Columns

public function getExportAdditionalColumns(): array
{
    return [
        'Total Visits' => fn($model) => $model->visits()->count(),
        'Created By' => fn($model) => $model->createdBy->name ?? 'N/A',
    ];
}

Custom Export Query

public static function getExportQuery(): Builder
{
    return static::query()
        ->with(['city', 'serviceArea'])
        ->where('status', 'active')
        ->orderBy('name');
}

Architecture

Core Components

• Contracts - Importable/Exportable interfaces
• Traits - HasImport/HasExport default implementations
• Enums - BatchStatus, ImportLogStatus for type-safe status values
• Data (DTOs) - ImportRequestData, ImportResultData, ExportRequestData, ExportResultData
• Models - ImportBatch, ImportLog, ExportBatch (with enum casting and DTO conversion)
• Services - ImportService, ExportService, Registry (return DTOs)
• Jobs - ProcessImportJob, ProcessExportJob, Cleanup (use enums)
• Events - Broadcasting for real-time updates
• Notifications - Email + Pusher notifications (export emails include file attachment)
• Controllers - REST API endpoints (use DTOs for validation and responses)

Status Values (Enums)

BatchStatus

• PENDING - Queued, not started
• PROCESSING - Currently running
• COMPLETED - Finished successfully
• FAILED - Failed with error

ImportLogStatus

• SUCCESS - Row imported successfully
• FAILED - Row failed validation/import
• SKIPPED - Row was empty or skipped

Data Transfer Objects (DTOs)

ImportRequestData

• Validates file upload (xlsx, xls, csv)
• Max size: 10MB (configurable)

ImportResultData

• Type-safe import response
• Properties: batch_id, status, message, progress, statistics, errors

ExportRequestData

• Validates export filters
• Supports: status, city_id, date ranges, search

ExportResultData

• Type-safe export response
• Properties: batch_id, status, file info, download_url, expiration

Email Notifications

Import Completed

• Subject: "Import Completed Successfully"
• Includes: Statistics, error count, link to view errors

Export Ready

• Subject: "Export Ready for Download"
• Includes: Download link, file size, expiration date
• Attachment: Excel file attached directly to email
• Expires: 7 days

Failure Notifications

• Sent for both import and export failures
• Includes error message and troubleshooting tips

File Structure

ims-backend/src/modules/ImportExport/
├── Contracts/          # Importable, Exportable interfaces
├── Traits/             # HasImport, HasExport
├── Enums/              # BatchStatus, ImportLogStatus
├── Data/               # DTOs (Request/Result)
├── Models/             # ImportBatch, ImportLog, ExportBatch
├── Services/           # ImportService, ExportService, Registry
├── Jobs/               # ProcessImportJob, ProcessExportJob, Cleanup
├── Events/             # Import/Export Started/Progress/Completed/Failed
├── Notifications/      # Email + Pusher notifications
├── Controllers/        # Import, Export, SampleData controllers
├── Migrations/         # Database tables
├── Providers/          # ImportExportServiceProvider
├── Config/             # import-export.php
└── README.md

License

MIT