bleksak / sqlite-toolkit
SQLite parser, AST and query builder for PHP
Maintainers
Requires (Dev)
- carthage-software/mago: ^1.47
- phpunit/phpunit: ^13.3
README
PHP parser for the SQLite 3.54.0 grammar: lexing, a
hand-written recursive-descent parser, and a typed AST. The grammar contract
lives in docs/spec/.
Benchmarks
Measured with tools/bench.php. The comparison workload
is a generated script of 10,000 statements (CREATE TABLE, INSERT … ON
CONFLICT, CTE + window-function SELECT, CASE UPDATE, subquery DELETE, in
round-robin order), run against the same 8,000 pre-created tables each round.
Setup time (≈1.7 s) is excluded; all times are best-of-rounds.
| Workload (identical 10,000-statement script) | Time | Throughput |
|---|---|---|
SQLite 3.53.4 — prepare() only (parse + compile, no execution, no AST) |
54 ms | 185k stmt/s |
| This project — parse into a full AST | 330 ms | 30.3k stmt/s |
SQLite 3.53.4 — exec() (parse + compile + execute) |
1,113 ms | 9k stmt/s |
Peak memory: 184 MB; retained ASTs per 10,000-statement run: 61 MB. Under the settings in Configuration the parser row is 223 ms, 44.8k stmt/s.
Measured with opcache on and nothing else, which is the floor PHP runs at wherever it is deployed. The tracing JIT and preloading are worth about a quarter between them and are not in this row — see Configuration.
About a fifth of that time is lexing and the rest the state machine.
Environment
| CPU | AMD Ryzen 7 9850X3D (8 cores / 16 threads, up to 5.65 GHz) |
| Motherboard | ASUSTeK ProArt X870E-CREATOR WIFI |
| RAM | 64 GB |
| OS | CachyOS Linux |
| Kernel | 7.2.0-1-cachyos (#1 SMP PREEMPT_DYNAMIC, built 2026-08-20) |
| PHP | 8.5.6 (ZTS) |
| SQLite | 3.53.4 (PHP-bundled, used by both SQLite rows) |
Configuration
Enable opcache, the tracing JIT and preloading. The parser is a hot loop of small calls and table lookups, which is the shape the JIT and preloading are both good at.
opcache.enable = 1 ; tracing only — function mode measures worse than no JIT at all opcache.jit = tracing opcache.jit_buffer_size = 256M opcache.preload = /path/to/vendor/bleksak/sqlite-parser/tools/preload.php ; only when php starts as root opcache.preload_user = www-data
| On top of opcache | Script row |
|---|---|
| tracing JIT | 330 ms to 278 ms |
| preloading | 330 ms to 297 ms |
| both | 330 ms to 223 ms |
Preloading is the less obvious of the two and worth as much. Compiling
one file at a time, the optimiser cannot see another class's constants: a
match over token kinds comes out as one comparison per arm rather than
a jump table, and the class types on parameters and properties bind at
first use instead of up front. Preloaded, everything is linked before
anything is optimised.
opcache itself no longer measures differently here with the flag on or off; it is on the list because the other two are opcache features and need it. It used to be worth 7%, and the parser has since moved to a shape its optimiser has little left to say about.
Both settings want a process that serves more than one thing. Under php-fpm or a worker the preload is paid when the server starts; for a one-shot CLI script it is about 17 ms of startup that nothing amortises, and the JIT never gets warm enough to pay for itself either.
just bench-tuned runs the benchmark under all three.
Coding work
Architecture of the code was done by me. Most of the implementation was done by Qwen 3.8 27B model hosted locally on 2xR9700. Optimization work was done by Opus 5. Nearly 100% of the code is AI generated.