README

The Ultimate Laravel Import/Export Framework

Tired of manual data entry nightmares and clunky import/export processes?
Dream of a world where your application effortlessly ingests and disgorges vast datasets with bulletproof validation and intelligent relationship management?

Your quest ends here—with OmniPorter.

OmniPorter is a meticulously engineered, battle-tested solution that injects your Eloquent models with unparalleled import and export superpowers.
Prepare to catapult your application's data fluency into the stratosphere.

This isn't merely about shuffling bits and bytes; it's about intelligent, resilient, and user-centric data orchestration.
OmniPorter harnesses the full might of Laravel's queueing system, trait-based architecture, dynamic class resolution, and advanced caching mechanisms to forge a system that:

✅ Ingests mammoth files asynchronously, keeping your UI snappy
✅ Validates data with surgical precision, catching errors before they corrupt your database
✅ Navigates complex model relationships like a seasoned architect
✅ Ensures peak performance even under heavy load
✅ Keeps your users fully informed with real-time notifications

Let’s embark on this journey to bestow your models with these remarkable capabilities—OmniPorter style.

🧱 I. The Foundational Pillars: Architecting OmniPorter’s Excellence

🧩 Traits (HasImport, HasExport)
Your plug-and-play powerups—just use them in your Eloquent models and you're off to the races.
📜 Interfaces (Importable, Exportable)
Sacred contracts that ensure any participating model defines the required metadata and methods.
🎩 Service Provider (ImportExportServiceProvider)
The dynamic scanner and registrar of all eligible models—auto-wiring FTW, courtesy of OmniPorter.
🧰 Controllers (ImportController, ExportController)
OmniPorter’s API gatekeepers: enforcing permissions, handling requests, and delegating heavy work to jobs.
🧠 Intelligent Caching (ImportDetailsCache, ExportDetailsCache)
Persist chunked job state in Redis for smooth recovery and flawless continuity.
📦 Queues & Jobs
All major processing (reading, saving, exporting) happens off the main thread—no UI freeze ever.
📬 Notifications
Clear and concise emails are dispatched to notify users of success or failure—with full results.

📥 II. Making Your Model Importable with OmniPorter

🔧 A. Implementing the `Importable` and `Exportable` Contracts

use OmniPorter\Contracts\Importable;
use OmniPorter\Contracts\Exportable;
use Illuminate\Database\Eloquent\Model;

class YourModel extends Model implements Importable, Exportable
{
    public static function getUniqueKeyForImportExport(): string
    {
        return 'work_email';
    }

    public static function getListOfRelationDetails(): array
    {
        return [
            Department::class => ['type' => 'belongsTo', 'method' => 'department', 'field' => 'department_id'],
            Employee::class => ['type' => 'belongsTo', 'method' => 'manager', 'field' => 'reports_to'],
            Profile::class => ['type' => 'hasOne', 'method' => 'profile', 'field' => 'profile_id'],
            Role::class => ['type' => 'belongsToMany', 'method' => 'roles'],
        ];
    }
}

✨ B. Leveraging the `HasImport` Trait

use OmniPorter\Traits\HasImport;

class YourModel extends Model implements Importable, Exportable
{
    use HasImport;
}

✅ C. Model-Specific Validation

// CreateEmployeeRequest
public function rules(): array
{
    return [
        'work_email' => ['required', 'string', 'email', 'max:255', 'unique:employees,work_email'],
        // ...
    ];
}

// UpdateEmployeeRequest
public function rules(?int $id = null): array
{
    return [
        'work_email' => ['required', 'string', 'email', 'max:255', Rule::unique('employees', 'work_email')->ignore($id)],
        // ...
    ];
}

🔄 D. Mapping Excel Headings to Model Fields

Excel headers should match your model’s fillable fields.
For belongsTo relations, use the related model's getUniqueKeyForImportExport() return value as the header name.
For belongsToMany relations, provide a comma-separated list of unique keys.

Note

hasOne relations are currently supported for Export only. For Import, use belongsTo on the model that holds the foreign key.

🔮 E. Auto-Creating Missing Related Records

OmniPorter can automatically create missing related records during import. For example, if an employee's CSV contains a designation that doesn't exist yet, OmniPorter can create it on the fly.

To enable this, implement the getAutoCreateAttributesOnImport() static method on the related model:

class Designation extends Model
{
    /**
     * Return the attributes for auto-creating this model during import.
     * Return null to disable auto-creation.
     */
    public static function getAutoCreateAttributesOnImport(string $value): ?array
    {
        return [
            'title' => $value,       // The matched column
            'type'  => 'employee',   // Default values for required fields
        ];
    }
}

How it works:

During import, if a belongsTo relation value (e.g., "Data Scientist") is not found in the database, OmniPorter checks if the related model has getAutoCreateAttributesOnImport().
If the method exists and returns a non-null array, a new record is created with those attributes.
The import cache is updated so subsequent rows referencing the same value don't trigger duplicate creation.
All auto-creations are logged for audit purposes.

Tip

If getAutoCreateAttributesOnImport() returns null, the relation is skipped (set to null) and a warning is logged. This allows you to opt-in per model.

📤 III. Making Your Model Exportable with OmniPorter

🧲 A. Use the `HasExport` Trait

use OmniPorter\Traits\HasExport;

class YourModel extends Model implements Importable, Exportable
{
    use HasExport;
}

📃 B. Define `getColumnsToExport()`

public static function getColumnsToExport(): array
{
    return [
        'first_name',
        'work_email',
        'department',
        'roles',
    ];
}

🔍 C. Filtering Exports the OmniPorter Way

Example queries:

?status=active
?name_like=john
?department_id_in=1,2,3

Suffix	Description
`_eq`	Equal to
`_ne`	Not equal to
`_gt`	Greater than
`_gte`	Greater than or equal to
`_lt`	Less than
`_lte`	Less than or equal to
`_like`	SQL LIKE (e.g. `%value%`)
`_nlike`	SQL NOT LIKE
`_in`	In list (comma-separated)
`_nin`	Not in list (comma-separated)
`_null`	Is NULL
`_nnull`	Is NOT NULL

🌐 IV. Routing Data Traffic with OmniPorter

📥 Import Endpoint

POST /imports/{resource}/{mode}

mode: create or update
file: required in body

📤 Export Endpoint

GET /exports/{resource}

Fully filterable
OmniPorter handles everything in the background

📊 Progress Tracking Endpoint

GET /imports/progress/{batchId}

Track the progress of an import in real-time:

Response:

{
  "success": true,
  "data": {
    "batch_id": "550e8400-e29b-41d4-a716-446655440000",
    "total_rows": 1000,
    "processed_rows": 750,
    "progress": 75.0,
    "status": "in_progress"
  }
}

Status Values:

pending: Import has not started
in_progress: Import is currently processing
completed: Import has finished

WebSocket Events: For real-time updates, listen to the omniporter-progress.{batchId} channel:

Echo.channel(`omniporter-progress.${batchId}`)
    .listen('ProgressUpdated', (e) => {
        console.log(`Progress: ${e.progress}%`);
        console.log(`Processed: ${e.processedRows}/${e.totalRows}`);
    });

🛡️ V. OmniPorter Authorization & Permissions

employee:import, employee:create, employee:store
employee:update, employee:edit
employee:export, employee:index

$user->hasAnyPermission(['employee:create', 'employee:store']);

⚙️ VI. Configuration

Publish the config file:

php artisan vendor:publish --tag="omniporter-config"

Key	Environment Variable	Default	Description
`cache.store`	`OMNIPORTER_CACHE_STORE`	`redis`	The cache store for job state.
`cache.prefix`	`OMNIPORTER_CACHE_PREFIX`	`omniporter`	Prefix for cache keys.
`cache.ttl`	`OMNIPORTER_CACHE_TTL`	`3600`	How long to keep state (seconds).
`import.disk`	`OMNIPORTER_IMPORT_DISK`	`local`	Disk for uploads/results.
`export.disk`	`OMNIPORTER_EXPORT_DISK`	`local`	Disk for final export files.

🧹 VII. Maintenance

Clearing Cache

If you encounter issues with stuck jobs or state, you can clear the OmniPorter cache:

php artisan omniporter:clear-cache

🏗️ VIII. Behind the Curtains of OmniPorter: How it Works

OmniPorter is designed for massive scale and reliability. Here is how the magic happens:

📥 The Import Flow

Upload: The user sends a file to the ImportController.
Storage: The file is stored on the configured disk.
Queueing: A GenericImport job is dispatched to the queue.
Processing:
- The file is read in chunks using Maatwebsite\Excel.
- Each row is validated using your model's getImportValidators().
- Relationships are resolved via getListOfRelationDetails().
- Records are created or updated using the unique key from getUniqueKeysForUpdate().
State Tracking: Progress and failures are tracked in real-time using the ImportDetailsCache (stored in Redis).
Completion: A notification is sent to the user with a summary of successes and failures.

📤 The Export Flow

Request: The user requests an export via ExportController, optionally providing filters and columns.
Filtering: OmniPorter applies dynamic filters based on the request parameters.
Queueing: A GenericExport job is dispatched.
Generation:
- Data is queried in chunks to avoid memory issues.
- Relationships are eager-loaded for performance.
- The final spreadsheet is generated and saved to the export disk.
Notification: The user receives an email with a secure download link.

🎯 Conclusion: OmniPorter, Your Data Swiss Army Knife

You now wield OmniPorter—a high-performance Laravel-based import/export engine that’s:

✅ Asynchronous
✅ Resilient
✅ Secure
✅ Developer-Friendly

Go forth, and automate with elegance—with OmniPorter.
Because your data deserves better.

deep-shah / omniporter

Maintainers

Package info

Statistics

Security

README

The Ultimate Laravel Import/Export Framework

🧱 I. The Foundational Pillars: Architecting OmniPorter’s Excellence

📥 II. Making Your Model Importable with OmniPorter

🔧 A. Implementing the `Importable` and `Exportable` Contracts

✨ B. Leveraging the `HasImport` Trait

✅ C. Model-Specific Validation

🔄 D. Mapping Excel Headings to Model Fields

🔮 E. Auto-Creating Missing Related Records

📤 III. Making Your Model Exportable with OmniPorter

🧲 A. Use the `HasExport` Trait

📃 B. Define `getColumnsToExport()`

🔍 C. Filtering Exports the OmniPorter Way

🌐 IV. Routing Data Traffic with OmniPorter

📥 Import Endpoint

📤 Export Endpoint

📊 Progress Tracking Endpoint

🛡️ V. OmniPorter Authorization & Permissions

⚙️ VI. Configuration

🧹 VII. Maintenance

Clearing Cache

🏗️ VIII. Behind the Curtains of OmniPorter: How it Works

📥 The Import Flow

📤 The Export Flow

🎯 Conclusion: OmniPorter, Your Data Swiss Army Knife

deep-shah / omniporter

Maintainers

Package info

Statistics

Security

README

The Ultimate Laravel Import/Export Framework

🧱 I. The Foundational Pillars: Architecting OmniPorter’s Excellence

📥 II. Making Your Model Importable with OmniPorter

🔧 A. Implementing the Importable and Exportable Contracts

✨ B. Leveraging the HasImport Trait

✅ C. Model-Specific Validation

🔄 D. Mapping Excel Headings to Model Fields

🔮 E. Auto-Creating Missing Related Records

📤 III. Making Your Model Exportable with OmniPorter

🧲 A. Use the HasExport Trait

📃 B. Define getColumnsToExport()

🔍 C. Filtering Exports the OmniPorter Way

🌐 IV. Routing Data Traffic with OmniPorter

📥 Import Endpoint

📤 Export Endpoint

📊 Progress Tracking Endpoint

🛡️ V. OmniPorter Authorization & Permissions

⚙️ VI. Configuration

🧹 VII. Maintenance

Clearing Cache

🏗️ VIII. Behind the Curtains of OmniPorter: How it Works

📥 The Import Flow

📤 The Export Flow

🎯 Conclusion: OmniPorter, Your Data Swiss Army Knife

🔧 A. Implementing the `Importable` and `Exportable` Contracts

✨ B. Leveraging the `HasImport` Trait

🧲 A. Use the `HasExport` Trait

📃 B. Define `getColumnsToExport()`