oliverde8 / php-etl-bundle
Allow usage of the PHP-ETL library in symfony framework.
Installs: 3 433
Dependents: 2
Suggesters: 0
Security: 0
Stars: 5
Watchers: 2
Forks: 2
Open Issues: 3
Requires
- php: >=8.0
- ext-json: *
- oliverde8/php-etl: ^v1.2.0-alpha1
- symfony/framework-bundle: ^5.4|^6.0|^7.0
- symfony/messenger: ^5.4|^6.0|^7.0
Requires (Dev)
- symfony/console: ^6.0|^7.0
- symfony/monolog-bundle: ^3.7
This package is auto-updated.
Last update: 2024-10-24 13:16:06 UTC
README
The Php etl bundle allows the usage of Oliver's PHP Etl library in symfony.
You should also check the PHP ETL's Easy Admin Bundle to have interfaces.
Installation
-
Install using composer
-
in
/config/
create a directoryetl
-
Enable bundle:
\Oliverde8\PhpEtlBundle\Oliverde8PhpEtlBundle::class => ['all' => true],
- Optional: Enable queue if you wish to allow users from the easy admin panel to do executions.
framework: messenger: routing: "Oliverde8\PhpEtlBundle\Message\EtlExecutionMessage": async
- Optional: Enable creation of individual files for each log by editing the monolog.yaml
etl: type: service id: Oliverde8\PhpEtlBundle\Services\ChainExecutionLogger level: debug channels: ["!event"]
Usage
Creating an ETL chain
First read the documentation of the PHP ETL
Each chain is declare in a single file. The name of the chain is the name of the file created in /config/etl/
.
Example:
chain: "Dummy Step": operation: rule-engine-transformer options: add: true columns: test: rules: - get : {field: [0, 'uid']}
Executing a chain
./bin/console etl:execute demo '[["test1"],["test2"]]' '{"opt1": "val1"}'
The first argument is the input, depending on your chain it can be empty. The second are parameters that will be available in the context of each link in the chain.
Additional commands
Get a definition
./bin/console etl:get-definition demo
Adding your own chain operation
To add your own chain operation you need 2 classes. The operation itself that we will call
MyVendor\Etl\Operation\OurTestOperation
, and a MyVendor\Etl\OperationFactory\OurTestOperationFactory
factory
to create it. The factory allows us to configure the operation and inject service to our operation.
All operations needs to implement DataChainOperationInterface
; they can extend AbstractChainOperation
.
All factories needs to extend Oliverde8\Component\PhpEtl\Builder\Factories\AbstractFactory
.
The operation is a Model and not a service, you therefore need to add the path to the exclusions so that it's not made a service by symfony:
App\: resource: '../src/' exclude: - '../src/Etl/Operation'
Factories needs to be tagged `etl.operation-factory\ . To remove the need to tag all your factories you can add the following line your your services.yaml file
MyVendor\Etl\OperationFactory\: resource: '../src/Etl/OperationFactory/' tags: ['etl.operation-factory']
For more information on how the etl works and how to create operations check the Php Etl Documentation