README

Pipes

Pipes is a PHP Extract Transform Load [ETL] package for Laravel 8+

Installation

composer require jwhulette/pipes

Usage

Create a new EtlPipe object.
Add an extractor to the object to read the input file
- You can create your own extractor by implementing the ExtractorInterface.php
Add transformers to transform the data
- You can add as many transformers as you want.
- You can create your own transformers by implementing the TransformerInterface.php
- Data is passed to the transformers in the order they are defined
Add a loader to write out the transformed data.
- You can create your own loader by implementing the LoaderInterface.php

(new EtlPipe())
->extract(new CsvExtractor($this->csvFile));
->transforms([
    new CaseTransformer([], 'lower'),
    new TrimTransformer(),
])
->load(new CsvLoader('saved-file.csv'));

Notes:

Built-in extractors:

CsvExtractor
XlsxExtractor
SqlExtractor

Built-in loaders:

CsvLoader
SqlLoader

Built-in transformers:

CaseTransformer - Change the case of a string
DateTimeTransformer - Change the format of a date string
PhoneTransformer - Transform a US phone, removing all non numeric characters, and limiting the length to the first 10 digits
TrimTransformer - Trim a string
ZipcodeTransformer - Transform a US zip code, removing all non numeric characters, and left pad zeros for zip codes less than 5 digits
ConditionalTransformer - Transform a column, based on the values of another column

Data is passed to the transformers in the order they are defined

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Credits

License

The MIT License (MIT). Please see License File for more information.

jwhulette / pipes

Maintainers

Details