fruivita/line-reader

Read large files, line by line, without causing memory overflow for Laravel applications

v1.0.0 2022-05-10 02:14 UTC

Requires

• php: ^8.0
• illuminate/collections: ^9.11
• illuminate/pagination: ^9.11
• illuminate/support: ^9.11

Requires (Dev)

• nunomaduro/larastan: ^2.1
• orchestra/testbench: ^7.4
• pestphp/pest: ^1.21
• pestphp/pest-plugin-laravel: ^1.2
• phpstan/phpstan: ^1.6
• phpstan/phpstan-deprecation-rules: ^1.0
• phpunit/phpunit: ^9.5

MIT

• Fábio Cassiano

• php
• laravel
• reader
• file-reader
• fruivita
• line-reader

README

This package, for Laravel applications, allows you to read the contents of huge files without killing your server, that is, without having to load all the contents at once in memory causing an out-of-memory errors.

The strategy used here is to read the contents of the file, line by line, optimizing the use of server resources and, most importantly, in an efficient way.

It is also possible to paginate the contents of the file, again, without having to load it entirely into memory, except the page itself.

use FruiVita\LineReader\Facades\LineReader;

$generator = LineReader::readLines($file_path);

// or

$length_aware_paginator = LineReader::readPaginatedLines($file_path, $per_page, $page);

 

Table of Contents

1. Notes

2. Prerequisites

3. Installation

4. How it works

5. Testing and Continuous Integration

6. Changelog

7. Contributing

8. Code of conduct

9. Security Vulnerabilities

10. Support and Updates

11. Roadmap

12. Credits

13. Thanks

14. License

Notes

⭐ Internally, this package reads the file contents using php's SplFileObject and Generators classes. In the specific case of pagination, the LimitIterator class is used to delimit the beginning and end of the content to be read.

❤️ Heavily inspired by the bcremer/LineReader package.

⬆️ Back

 

Prerequisites

1. PHP dependencies

   PHP ^8.0

   Extensions

   composer check-platform-reqs

2. GitHub Package Dependencies

⬆️ Back

 

Installation

1. Install via composer:

   composer require fruivita/line-reader

2. Optionally publish the translations

   php artisan vendor:publish --provider='FruiVita\LineReader\LineReaderServiceProvider' --tag='lang'

   The strings available for translation are as follows. Change them as needed.

   {
       "The file entered could not be read": "The file entered could not be read"
   }

   This package already has translations for en and pt-br.

⬆️ Back

 

How it works

1. Reading a file line by line.

   use FruiVita\LineReader\Facades\LineReader;
   
   public function example()
   {
       foreach (LineReader::readLines($file_path) as $key => $line)
       {
           // $key is 0 when reading the 1st line, 1 when reading the 2nd line, and so on.
           // $line is a string with the contents of the line.
       }
   }

    

   LineReader exposes the following method to read the file line by line:

   ✏️ readLines

   use FruiVita\LineReader\Facades\LineReader;
   
   /**
    * @param string $file_path full path of the file to be read
    * 
    * @throws \FruiVita\LineReader\Exceptions\FileNotReadableException
    *
    * @return \Generator
    */
   LineReader::readLines(string $file_path);

    

   🚨 Exceptions:

   • readLines throws \FruiVita\LineReader\Exceptions\FileNotReadableException if don't have read permission on the file or it can't be found

    

2. Reading the file by page.

   use FruiVita\LineReader\Facades\LineReader;
   
   public function example()
   {
       $per_page = 15;
       $page = 2;
   
       $length_aware_paginator = LineReader::readPaginatedLines(string $file_path, int $per_page, int $page);
       
       // The index of the items in the collection respects their position in the file using a zero-based index,
       // that is, in the example above the 1st item on page 2 will have index 15, since it is the 16th line of
       // the file and the last item on page 2 will have index 29, since it is the 30th line of the file.
   }

    

   LineReader exposes the following method to read the file by page:

   ✏️ readPaginatedLines

   use FruiVita\LineReader\Facades\LineReader;
   
   /**
    * @param string $file_path full path of the file to be read
    * @param int    $per_page
    * @param int    $page
    * @param string $page_name
    *
    * @throws \FruiVita\LineReader\Exceptions\FileNotReadableException
    * @throws \InvalidArgumentException
    * 
    * @return \Illuminate\Pagination\LengthAwarePaginator
    */
   LineReader::readPaginatedLines(string $file_path, int $per_page, int $page, string $page_name = 'page');

    

   🚨 Exceptions:

   • readPaginatedLines throws \FruiVita\LineReader\Exceptions\FileNotReadableException if don't have read permission on the file or it can't be found
   • readPaginatedLines throws \InvalidArgumentException if per_page or page is less than 1

⬆️ Back

 

Testing and Continuous Integration

composer analyse
composer test
composer coverage

⬆️ Back

 

Changelog

Please see CHANGELOG for more information on what has changed in each version.

⬆️ Back

 

Contributing

Please see CONTRIBUTING for more details on how to contribute.

⬆️ Back

 

Code of conduct

To ensure that everyone is welcome to contribute to this open-source project, please read and follow the Code of Conduct.

⬆️ Back

 

Security Vulnerabilities

Please see security policy how to report security vulnerabilities or flaws.

⬆️ Back

 

Support and Updates

The latest version will receive support and updates whenever the need arises. The others will receive updates for 06 months after being replaced by a new version and then discontinued.

Version	PHP	Release	End of Life
1.0	^8.0	09-05-2022	dd-mm-yyyy

🐛 Found a bug?!?! Open an issue.

⬆️ Back

 

Roadmap

✨ Any new ideas?!?! Start a discussion.

The following list contains identified and approved improvement needs that will be implemented in the first window of opportunity.

• n/a

⬆️ Back

 

Credits

• Fábio Cassiano

• All Contributors

⬆️ Back

 

Thanks

👋 Thanks to the people and organizations below for donating their time to build the open-source projects that were used in this package.

• ❤️ Laravel for the packages:

  • illuminate/collections

  • illuminate/pagination

  • illuminate/support

• ❤️ Orchestra Platform for the package orchestral/testbench

• ❤️ FriendsOfPHP for the package FriendsOfPHP/PHP-CS-Fixer

• ❤️ Nuno Maduro for the package nunomaduro/larastan

• ❤️ PEST for the packages:

  • pestphp/pest

  • pestphp/pest-plugin-laravel

• ❤️ Sebastian Bergmann for the package sebastianbergmann/phpunit

• ❤️ PHPStan for the packages:

  • phpstan/phpstan

  • phpstan/phpstan-deprecation-rules

• ❤️ ergebnis for the package ergebnis/composer-normalize

• ❤️ Shivam Mathur for the Github Action shivammathur/setup-php

• ❤️ GP for the Github Action paambaati/codeclimate-action

• ❤️ Stefan Zweifel for the Github Actions:

  • stefanzweifel/git-auto-commit-action

  • stefanzweifel/changelog-updater-action

💸 Some of these people or organizations have some products/services that can be purchased. If you can help them by buying one of them or becoming a sponsor, even for a short period, you will help the entire open-source community to continue developing solutions for everyone.

⬆️ Back

 

License

The MIT License (MIT). Please see the License File for more information.

⬆️ Back