README

English | 中文

A Symfony bundle for managing asynchronous data export tasks with DQL support, progress tracking, and user-based task management.

Features

Asynchronous Processing: Handle large data exports without blocking the main application
DQL Query Support: Use Doctrine Query Language for flexible and powerful data queries
Progress Tracking: Real-time monitoring of export progress with total and processed counts
User Management: Associate export tasks with authenticated users for security
Flexible Configuration: JSON-based column configuration for customizable exports
Error Handling: Comprehensive exception tracking and error reporting
Memory Monitoring: Track memory usage to optimize performance
Task Validation: Built-in validation and status management system
Audit Trail: Full tracking with timestamps and user attribution

Installation

composer require tourze/async-export-bundle

Requirements

PHP 8.1 or higher
Symfony 6.4 or higher
Doctrine ORM 3.0 or higher
Doctrine DBAL 4.0 or higher

Quick Start

<?php

use AsyncExportBundle\Entity\AsyncExportTask;
use Doctrine\ORM\EntityManagerInterface;
use Symfony\Component\Security\Core\Security;

// Create a new export task
$task = new AsyncExportTask();
$task->setDql('SELECT u FROM App\Entity\User u WHERE u.active = true');
$task->setEntityClass('App\Entity\User');
$task->setColumns(['id', 'email', 'name', 'createTime']);
$task->setFile('users_export.csv');
$task->setRemark('Export active users');
$task->setUser($security->getUser()); // Associate with current user
$task->setValid(true);

// Save the task
$entityManager->persist($task);
$entityManager->flush();

// The task is now ready for background processing
// Use Symfony Messenger or similar for async execution

Configuration

Bundle Registration

Add the bundle to your config/bundles.php:

<?php

return [
    // ...
    AsyncExportBundle\AsyncExportBundle::class => ['all' => true],
];

Service Configuration

The bundle automatically configures services in services.yaml:

# config/services.yaml or in your bundle
services:
    AsyncExportBundle\Repository\:
        resource: '../src/Repository/'

Entity Mapping

Ensure Doctrine is configured to scan the bundle entities:

# config/packages/doctrine.yaml
doctrine:
    orm:
        mappings:
            AsyncExportBundle:
                type: attribute
                dir: '%kernel.project_dir%/vendor/tourze/async-export-bundle/src/Entity'
                prefix: 'AsyncExportBundle\Entity'

Entity Structure

The AsyncExportTask entity provides comprehensive export task management:

Core Properties

user: Associated user (UserInterface)
file: Export file path (string, max 1000 chars)
entityClass: Target entity class name for EntityManager lookup
dql: Doctrine Query Language string for data selection
columns: JSON array defining export column configuration

Tracking Properties

totalCount: Total number of records to export
processCount: Number of records already processed
memoryUsage: Memory consumption during processing
valid: Task validation status flag

Metadata Properties

remark: Human-readable task description
exception: Error message storage for failed tasks
json: Additional parameter storage

Inherited Traits

SnowflakeKeyAware: Unique ID generation
TimestampableAware: Created/updated timestamps
BlameableAware: User tracking for audit trails

Repository Usage

<?php

use AsyncExportBundle\Entity\AsyncExportTask;
use AsyncExportBundle\Repository\AsyncExportTaskRepository;
use Doctrine\ORM\EntityManagerInterface;

// Get repository
$repository = $entityManager->getRepository(AsyncExportTask::class);

// Find tasks by user
$userTasks = $repository->findBy(['user' => $user]);

// Find valid/pending tasks
$validTasks = $repository->findBy(['valid' => true]);

// Find tasks by entity class
$entityTasks = $repository->findBy(['entityClass' => 'App\Entity\User']);

// Find tasks with progress tracking
$incompleteTasks = $repository->createQueryBuilder('t')
    ->where('t.processCount < t.totalCount')
    ->andWhere('t.valid = true')
    ->getQuery()
    ->getResult();

Advanced Usage

Progress Tracking

// Update task progress during processing
$task->setTotalCount(1000);
$task->setProcessCount(250); // 25% complete
$task->setMemoryUsage(memory_get_usage());
$entityManager->flush();

Error Handling

try {
    // Process export
    processExport($task);
} catch (\Exception $e) {
    $task->setException($e->getMessage());
    $task->setValid(false);
    $entityManager->flush();
}

Security

Access Control

The bundle integrates with Symfony's security component to ensure proper access control:

All export tasks are associated with authenticated users
Users can only access their own export tasks
Task creation requires valid authentication
File paths should be validated to prevent directory traversal

Data Protection

Export files should be stored in secure locations
Sensitive data in DQL queries should be properly escaped
Regular cleanup of completed export files is recommended
Memory usage tracking helps prevent resource exhaustion

Best Practices

Always validate user input for DQL queries
Implement proper file permission controls
Use secure file naming conventions
Monitor export task execution for anomalies
Implement rate limiting for export task creation

Console Commands

The bundle provides the following console commands for managing export tasks:

`async-export:process`

Process a single export task by ID.

Usage:

php bin/console async-export:process <task-id> [--force]

Arguments:

task-id: The ID of the export task to process (required)

Options:

--force: Force process the task even if it's already been processed

Examples:

# Process task with ID 123
php bin/console async-export:process 123

# Force reprocess task with ID 456
php bin/console async-export:process 456 --force

`async-export:process-all`

Process all pending export tasks in the queue.