README

Memory-efficient and fast streaming JSON parser.

Example: parsing GeoJson from a gzipped JSON file containing a FeatureCollection, streaming it feature by feature. The second parameter to JsonStreamer constructor is the path in your JSON structure that contains the array that you want streamed. In this example, GeoJson's FeatureCollection is an object that has a 'features' property with an array of feature objects.

$fp = fopen("compress.zlib://" . __DIR__ . "/Fiji.GeoJson.gz", "r");
$parser = new \JsonStreamer\JsonStreamer($fp, "features");
foreach ($parser as $val) {
    echo substr(json_encode($val), 0, 100)."\n\n";
}

The array that you'd like to stream can contained deeply nested in the JSON structure. Here, the data that we're interested in, are located at level 3. If you were to access this array in Javascript, that would be like this:

let container = {
    "lvl1": {
        "lvl2": {
            "lvl3": [
                [1, 2, 3],
                [1, 2, 3, 4],
                [1, 2, 3, 4, 5],
                [1, 2, 3, 4, 5, 6]
            ]
        }
    }
}
let data = container.lvl1.lvl2.lvl3

To iterate through the same data in PHP with JsonStreamer, do the following:

$fp = fopen('data://text/plain,' . $json,'r');
$parser = new \JsonStreamer\JsonStreamer($fp, "lvl1.lvl2.lvl3");
foreach ($parser as $val) {
    echo json_encode($val) . "\n\n";
}

JsonStreamer has been built with GeoJson in mind, and is only capable of iterating over JSON arrays (possibly nested inside wrapper objects). However, it does it's work quite efficiently, as compared to other streaming JSON parsers. I was able to parse huge GeoJson files containing enormous GeoJson shapes (multipolygons) tens times faster and using ~4 times less memory than any other streaming JSON parser I tried. I'm saying this without any intent to undermine those projects: they are generic JSON parsers while JsonStreamer only does a very specific kind of job.

crocodile2u / json-streamer

Maintainers

Details

README