README

TOON (Token-Oriented Object Notation) for Laravel — a spec-compliant encoder and decoder for the format that carries the JSON data model in far fewer tokens.

🇹🇷 Türkçe dokümantasyon

Why this exists

When you put data into an LLM prompt, you pay for every token. JSON spends a large share of them repeating the same keys on every element of an array and punctuating structure it already made obvious:

[
  {"id":1,"name":"Ada","role":"admin"},
  {"id":2,"name":"Bob","role":"user"}
]

TOON declares the shape once and then writes rows:

[2]{id,name,role}:
  1,Ada,admin
  2,Bob,user

Same data, same JSON data model, roughly 40 % fewer tokens on tabular payloads. Because each array declares its own length, a truncated or hallucinated response fails validation instead of quietly passing through — the property that makes TOON genuinely useful for model output, not just input.

This package is a complete PHP implementation of the TOON specification v4.1, wired into Laravel: a facade, helpers, a fluent builder, a response macro, content negotiation, a Blade directive, filesystem helpers and five Artisan commands.

Measured token savings

Real numbers from tiktoken, not estimates. Reproduce them with php benchmarks/generate.php && python3 benchmarks/tokens.py.

cl100k_base (GPT-4 / 3.5)

Dataset	JSON (minified)	JSON (pretty)	TOON (comma)	TOON (tab)	vs minified JSON
users (100 uniform rows)	3,007	5,107	1,752	1,695	−43.6 %
orders (60 rows, nested objects)	1,696	3,135	1,028	1,006	−40.7 %
config (deep, non-uniform)	64	127	78	79	+21.9 %
single record	22	35	21	21	−4.5 %

o200k_base (GPT-4o / o-series)

Dataset	JSON (minified)	JSON (pretty)	TOON (comma)	TOON (tab)	vs minified JSON
users (100 uniform rows)	2,983	5,083	1,740	1,682	−43.6 %
orders (60 rows, nested objects)	1,693	3,072	984	961	−43.2 %
config (deep, non-uniform)	65	127	78	79	+20.0 %
single record	22	35	21	21	−4.5 %

Read the third row. TOON is worse than minified JSON on deep, non-uniform data — indentation costs more than the braces it replaces when there is no repetition to eliminate. TOON pays off on arrays of objects that share the same fields. If your payload is a nested config tree, keep JSON.

Against pretty-printed JSON — which is what most people actually paste into a prompt — TOON wins everywhere, by roughly 2–3×.

Use php artisan toon:bench your-payload.json to measure your own data.

Installation

composer require digitalcorehub/laravel-toon

The service provider and Toon facade are auto-discovered. Publish the config only if you want to change the defaults:

php artisan vendor:publish --tag=toon-config

Requirements: PHP 8.3+ and Laravel 12 or 13.

Laravel 10 and 11 are past their security-fix windows (February 2025 and March 2026). Composer refuses to install them, and supporting them here would mean pulling an unpatched framework into your application.

Quick start

use DigitalCoreHub\Toon\Facades\Toon;

$toon = Toon::encode(User::query()->select('id', 'name', 'role')->get());
// [3]{id,name,role}:
//   1,Ada,admin
//   2,Bob,user
//   3,Cem,editor

$data = Toon::decode($toon);
// [['id' => 1, 'name' => 'Ada', 'role' => 'admin'], ...]

encode() accepts anything: arrays, Collections, Eloquent models, enums, DateTimeInterface, Arrayable, JsonSerializable, generators, or a JSON string.

Putting it in a prompt

$prompt = <<<TXT
Here are this week's orders. Answer only from this data.

{$toon}
TXT;

Validating model output

This is where the declared lengths earn their keep:

try {
    $rows = Toon::decode($llmResponse);
} catch (ToonDecodeException $e) {
    // "Header declares 12 row(s) but the scope contains 9 (SPEC §14.1) (line 4)"
    report($e);
}

A model that stops mid-table produces a document that fails to parse, rather than a shorter array you never notice.

The format in one page

# Objects: one field per line
id: 123
name: Ada
active: true

# Nested objects: indentation, no braces
address:
  city: Istanbul
  postcode: "34000"

# Primitive arrays: inline, length declared
tags[3]: admin,ops,dev

# Arrays of uniform objects: the tabular form, fields declared once
users[2]{id,name,role}:
  1,Ada,admin
  2,Bob,user

# Uniform nested objects collapse into field groups
orders[2]{id,customer{name,country},total}:
  1,Ada,DK,99
  2,Bob,UK,149

# Objects whose values share a shape: the keyed tabular form
stats[2:]{views,clicks}:
  monday: 100,12
  tuesday: 140,19

# Mixed or non-uniform arrays fall back to list form
items[3]:
  - 1
  - name: thing
  - [2]: a,b

# Empty array vs empty object — distinct, on purpose
nothing: []
blank:

Strings are quoted only when they must be — when empty, padded, numeric-looking, equal to true/false/null, or containing a delimiter, colon, quote, bracket or a leading -/#. That rule is what keeps the token count down while remaining unambiguous.

Laravel integration

Facade, helper and builder

use DigitalCoreHub\Toon\Facades\Toon;

Toon::encode($data);
Toon::decode($toon);
Toon::validate($toon);                 // bool, never throws
Toon::errorFor($toon);                 // the reason, or null
Toon::tryDecode($toon, default: []);   // decode or fall back

toon_encode($data);                    // global helpers
toon_decode($toon);
toon()->encode($data);                 // resolve the manager

// Fluent builder
Toon::from($users)->tabs()->indent(4)->encode();
Toon::fromToon($input)->lax()->objects()->decode();
Toon::fromJson('{"id":1}')->encode();

API responses

// Explicit
return response()->toon($orders);       // Content-Type: text/toon; charset=utf-8

// Or negotiate: return TOON only when the client asks for it
Route::middleware(NegotiateToon::class)->group(function () {
    Route::get('/orders', fn () => response()->json(Order::all()));
});

GET /orders
Accept: text/toon

Existing JSON clients are untouched; an LLM-facing consumer sends the header and gets the cheaper payload.

Downloads and storage

return Toon::download('orders-june', $orders);   // orders-june.toon attachment

Toon::store('reports/june', $data);              // -> toon/reports/june.toon
Toon::retrieve('reports/june');
Toon::exists('reports/june');
Toon::delete('reports/june');

Toon::toFile(storage_path('export.toon'), $data);   // plain filesystem
Toon::fromFile(storage_path('export.toon'));

Blade

@toon($order)      {{-- <pre class="toon">…</pre> --}}
@toonraw($order)   {{-- just the encoded text --}}

Logging

toon()->log($payload);                       // default channel, debug level
toon()->log($payload, 'warning', 'stack');

Large tables

Toon::encode() holds the whole document in memory. For a table that does not fit, stream it row by row:

use DigitalCoreHub\Toon\Streaming\StreamWriter;

$writer = StreamWriter::toFile(storage_path('events.toon'));
$writer->writeTable(Event::lazy(), count: Event::count(), key: 'events');
$writer->close();

Peak memory stays proportional to a single row.

Artisan commands

php artisan toon:encode data.json --stats            # convert, show savings
php artisan toon:encode data.json -o out.toon -d pipe
php artisan toon:decode data.toon -o out.json
php artisan toon:validate "storage/*.toon"           # exit code 1 on failure
php artisan toon:store data.json exports/june --disk=s3
php artisan toon:bench data.json                     # size + throughput vs JSON

All of them accept a file path, a disk:path reference, - for STDIN, or raw content:

cat data.json | php artisan toon:encode -

Debugbar

With barryvdh/laravel-debugbar installed you get a TOON tab listing every encode/decode with its duration and the size saved versus JSON. When Debugbar is absent the profiling code is a single null check.

Configuration

// config/toon.php
'encoder' => [
    'indent_size' => 2,          // spaces per level
    'delimiter'   => 'comma',    // comma | tab | pipe
],
'decoder' => [
    'indent_size' => 2,
    'strict'      => true,       // enforce the full SPEC §14 checklist
    'associative' => true,       // arrays rather than stdClass
    'max_depth'   => 512,
],

Any option can be overridden per call:

Toon::encode($data, ['delimiter' => 'tab']);
Toon::decode($input, ['strict' => false, 'associative' => false]);

On delimiters: tab is usually a token or two cheaper (see the table above) but is invisible in a terminal and fragile in anything that reflows whitespace. Comma is the safe interchange default; reach for tab when you are optimising a prompt and control both ends.

Strict mode

Strict decoding is on by default and enforces the specification's complete error checklist:

Rejected	Example
Length mismatch	`tags[3]: a,b`
Row width mismatch	a row with fewer cells than the header declares
Duplicate sibling keys	`a: 1` twice at the same depth
Bad indentation	tabs, or a non-multiple of `indent_size`
Depth jumps and stray over-indented lines
Blank line inside an array scope
Trailing content after a root array
Invalid escapes, unterminated strings, surrogate escapes	`"\ud800"`

['strict' => false] relaxes what the spec permits (duplicate keys resolve last-write-wins, counts are advisory). Two things stay errors in every mode, because ignoring them would silently lose data: a bare token inside a scope, and a missing colon in key position.

Every failure carries the source line:

catch (ToonDecodeException $e) {
    $e->sourceLine;   // 4
    $e->getMessage(); // "... (SPEC §14.1) (line 4)"
}

Catch ToonException to handle any failure from this package, or ToonEncodeException / ToonDecodeException for one direction.

PHP-specific behaviour

The specification requires implementations to document how they map the JSON data model onto host types. Here is ours.

Empty object vs empty array. PHP cannot express both with [], so the package follows json_encode()'s convention: a PHP list is a JSON array, any other array is a JSON object, and [] is an empty array. Pass new stdClass (or (object) []) when you mean an empty object.

Toon::encode(['a' => [], 'b' => new stdClass]);
// a: []
// b:

Decoding shape. decode() returns associative arrays, which is what Laravel code wants — but PHP arrays cannot represent an empty object and coerce integer-like keys to integers. For an exact round trip use ['associative' => false] and get stdClass back.

Numeric domain. Integers outside PHP's range decode to floats, matching json_decode(). Encoding large integers from a JSON string keeps them lossless by emitting a quoted string (JSON_BIGINT_AS_STRING), as SPEC §2 permits.

Tab indentation. An error in strict mode. In non-strict mode each leading tab counts as one level and remaining spaces contribute floor(spaces / indent_size).

Prototype keys. __proto__, constructor and friends are ordinary keys here; PHP arrays hold them safely.

Invalid UTF-8 is rejected rather than replaced with U+FFFD.

Specification conformance

The package is validated against the official toon-format/spec fixture suite, vendored into tests/Fixtures/spec/ so the suite runs offline:

Tests:  875 passed
        ├─ 538 official spec fixtures (encode + decode + error cases)
        ├─ spec conversion examples and valid/invalid documents
        ├─ property round-trip tests over a pseudo-random corpus
        └─ Laravel integration
PHPStan: level 8, no errors
Coverage: 96.7 %

composer test        # Pest
composer analyse     # PHPStan
composer format      # Pint
composer check       # all three

Upgrading from 0.x

v1.0.0 changes the output format. Versions before 1.0 emitted a custom key, value; format that was not TOON and could not interoperate with any other implementation — and lost data on several shapes. v1.0.0 replaces it with the real specification.

Read UPGRADE.md before upgrading. Anything stored as .toon by 0.x must be re-encoded from its JSON source.

Contributing

Bug reports and pull requests are welcome — see CONTRIBUTING.md. Run composer check before opening a PR. If a change touches the encoder or decoder, it needs a test that would have failed before it.

Credits

The TOON format and its specification are the work of Johann Schopplich — toon-format/spec (MIT).
This package by Batuhan Haymana / DigitalCoreHub.

License

MIT — see LICENSE.

digitalcorehub / laravel-toon

Maintainers

Package info

Statistics

Security