joelwurtz/json-stream

Lazy, streaming JSON decoding: json_stream_decode() returns an array-accessible document that only parses the paths you access, with SIMD-accelerated scanning and constant-memory transient iteration

dev-main 2026-07-23 13:33 UTC

Requires

• php: >= 8.4

MIT

• Joel Wurtz

• json
• stream
• parser
• extension
• streaming
• lazy
• simd

Published on Packagist.org on 2026-07-23 13:41 UTC

Extension name: ext-json_stream

Supports Zend Thread Safety (ZTS): ✅

Supports non-Thread Safe mode (NTS): ✅

Supported operating systems: bsd, darwin, solaris, linux, unknown

README

PHP extension for lazy, streaming JSON decoding.

$data = json_stream_decode($source, $options);
echo $data['foo']['bar'];

$source may be a string, a stream resource, an array of string chunks, or any Traversable yielding string chunks (e.g. a Generator). Nothing is read from the source until the first access on the returned JsonStream\Document.

$options accepts the relevant JSON_* flags (JSON_BIGINT_AS_STRING, JSON_INVALID_UTF8_IGNORE, JSON_INVALID_UTF8_SUBSTITUTE) plus:

• JSON_STREAM_TRANSIENT — forward-only streaming: foreach over a container releases consumed input as it goes, so arbitrarily large streams iterate in constant memory. Once iteration starts the document is forward-only (random access and re-iteration throw).

$doc = json_stream_decode(fopen('huge.json', 'r'), JSON_STREAM_TRANSIENT);
foreach ($doc['rows'] ?? $doc as $row) {
    handle($row['id'], $row['payload']);
}

Status: the lazy engine is live — a resumable simdjson-style SIMD structural indexer (AVX2/SSSE3/SSE2 on x86_64, NEON/PMULL on aarch64, scalar fallback) feeds a three-state field model (unread → buffered span → parsed zval). Only the accessed path is ever materialized: sibling fields are recorded as byte spans, nested containers become lazy child documents sharing the parser state. See PLAN.md for the roadmap and the documented divergences from json_decode.

Install

With PIE (once published on Packagist):

pie install joelwurtz/json-stream

Or build from source:

phpize
./configure --enable-json_stream
make
NO_INTERACTION=1 make test

Polyfill

A pure-PHP polyfill of the full API lives in polyfill/: require polyfill/bootstrap.php (or use the package's Composer files autoload) and json_stream_decode() / JsonStream\Document work without the extension — much slower, but with the same lazy and constant-memory (transient) behavior. See polyfill/README.md for limits.