p1ratrulezzz/json-collection-parser

Streaming parser for large JSON files containing array of objects

1.5.4 2019-03-11 16:35 UTC

Requires

• php: ^7.1
• p1ratrulezzz/json-streaming-parser: ~6.0

Requires (Dev)

• phpunit/phpunit: ~7.0
• satooshi/php-coveralls: ~2.0

Suggests

• ext-zlib: Needed to support GZIP-compressed files

MIT

• Max Grigorian
• P1ratRuleZZZ

• json

README

Event-based parser for large JSON collections (consumes small amount of memory). Built on top of JSON Streaming Parser

This package is compliant with PSR-4, PSR-1, and PSR-2. If you notice compliance oversights, please send a patch via pull request.

Installation

You will need Composer to install the package

composer require p1ratrulezzz/json-collection-parser ~1.5

Input data format

Collection must be an array of objects.

[
    {
        "id": 78,
        "title": "Title",
        "dealType": "sale",
        "propertyType": "townhouse",
        "properties": {
            "bedroomsCount": 6,
            "parking": "yes"
        },
        "photos": [
            "1.jpg",
            "2.jpg"
        ]
    },
    {
        "id": 729,
        "dealType": "rent_long",
        "propertyType": "villa"
    },
    {
        "id": 5165,
        "dealType": "rent_short",
        "propertyType": "villa"
    }
]

Usage

Function as callback:

function processItem(array $item)
{
    is_array($item); //true
    print_r($item);
}

$parser = new \JsonCollectionParser\Parser();
$parser->parse('/path/to/file.json', 'processItem');

Closure as callback:

$items = [];

$parser = new \JsonCollectionParser\Parser();
$parser->parse('/path/to/file.json', function (array $item) use (&$items) {
    $items[] = $item;
});

Static method as callback:

class ItemProcessor {
    public static function process(array $item)
    {
        is_array($item); //true
        print_r($item);
    }
}

$parser = new \JsonCollectionParser\Parser();
$parser->parse('/path/to/file.json', ['ItemProcessor', 'process']);

Instance method as callback:

class ItemProcessor {
    public function process(array $item)
    {
        is_array($item); //true
        print_r($item);
    }
}

$parser = new \JsonCollectionParser\Parser();
$processor = new \ItemProcessor();
$parser->parse('/path/to/file.json', [$processor, 'process']);

Receive items as objects:

function processItem(\stdClass $item)
{
    is_array($item); //false
    is_object($item); //true
    print_r($item);
}

$parser = new \JsonCollectionParser\Parser();
$parser->parseAsObjects('/path/to/file.json', 'processItem');

Pass stream as parser input:

$stream = fopen('/path/to/file.json', 'r');

$parser = new \JsonCollectionParser\Parser();
$parser->parseAsObjects($stream, 'processItem');

Supported file formats

• .json - raw JSON format
• .gz - GZIP-compressed file (you will need zlib PHP extension installed)

Running tests

composer test

License

This library is released under MIT license.