README

The fastest TOON (Token-Oriented Object Notation) encoder and decoder for PHP, with full support for PHP 7.0 through 8.4.

TOON is a data serialization format optimized for LLM (Large Language Model) input, providing approximately 40% token reduction compared to JSON while maintaining full semantic equivalence.

⚡ Performance Highlights

This library is specifically optimized for speed and broad compatibility:

Feature	mfonte/fast-toon	helgesverre/toon
PHP Version	7.0 - 8.4 ✅	8.1+ only
Encoding Speed	3.2x faster	baseline
Decoding Speed	15x faster	baseline
Memory Usage	Optimized	Standard

Benchmark Results

Performance comparison against helgesverre/toon (the reference implementation):

Data Size	Operation	mfonte/fast-toon	helgesverre/toon	Speedup
Small (10 items)	Encode	9.4 ms	27.5 ms	2.9x
Small (10 items)	Decode	9.2 ms	33.5 ms	3.6x
Medium (100 items)	Encode	85 ms	284 ms	3.3x
Medium (100 items)	Decode	27 ms	456 ms	16.9x
Large (1000 items)	Encode	88 ms	274 ms	3.1x
Large (1000 items)	Decode	24 ms	463 ms	19.3x

Benchmarks run on PHP 8.3 with 1000 iterations (100 for large data). Results may vary based on hardware and PHP version.

Installation

You can install the package via composer:

composer require mfonte/fast-toon

Usage

use Mfonte\FastToon\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'active' => true],
        ['id' => 2, 'name' => 'Bob', 'active' => false],
        ['id' => 3, 'name' => 'Charlie', 'active' => true],
    ]
];

$toon = toon_encode($data);

// Output:
// users[3]{id,name,active}:
// 1|Alice|T
// 2|Bob|F
// 3|Charlie|T

$decoded = toon_decode($toon);

Fluent API

Instead of using helper functions, you may also use the fluent API:

use Mfonte\FastToon\Toon;

$toon = Toon::encode($data)
    ->withFlags(TOON_THROW_ON_ERROR | TOON_KEY_FOLDING)
    ->withDepth(100)
    ->toString();

$decoded = Toon::decode($toon)
    ->withOptions(['strict' => true])
    ->toArray();

Encoding flags

Like json_encode(), TOON supports bitmask flags for controlling encoding behavior:

// Combine flags with bitwise OR
$toon = toon_encode($data, TOON_THROW_ON_ERROR | TOON_KEY_FOLDING);

// Use tab delimiter for tabular output
$toon = toon_encode($data, TOON_TAB_DELIMITER);

Flag	Description
`TOON_THROW_ON_ERROR`	Throw exception instead of returning `false`
`TOON_FORCE_OBJECT`	Force empty/sequential arrays as objects
`TOON_PRESERVE_ZERO_FRACTION`	Preserve `.0` in floats like `5.0`
`TOON_TAB_DELIMITER`	Use tab as tabular delimiter
`TOON_PIPE_DELIMITER`	Use pipe as tabular delimiter (default)
`TOON_KEY_FOLDING`	Enable key folding for nested single-key objects
`TOON_INDENT_4`	Use 4 spaces for indentation
`TOON_NUMERIC_CHECK`	Encode numeric strings as numbers
`TOON_INVALID_UTF8_IGNORE`	Ignore invalid UTF-8 sequences
`TOON_INVALID_UTF8_SUBSTITUTE`	Substitute invalid UTF-8 with replacement char

Decoding flags

$data = toon_decode($toon, TOON_STRICT | TOON_THROW_ON_ERROR);

Flag	Description
`TOON_THROW_ON_ERROR`	Throw exception instead of returning `null`
`TOON_OBJECT_AS_ARRAY`	Decode objects as associative arrays
`TOON_BIGINT_AS_STRING`	Decode big integers as strings
`TOON_STRICT`	Enable strict mode validation

Error handling

TOON provides error handling similar to JSON:

$result = toon_encode($data);

if ($result === false) {
    echo toon_last_error_msg();
}

When using TOON_THROW_ON_ERROR, exceptions are thrown instead:

try {
    $toon = toon_encode($data, TOON_THROW_ON_ERROR);
} catch (\Mfonte\FastToon\Exceptions\EncodingException $e) {
    // Handle exception
}

TOON format overview

TOON uses a line-based syntax that eliminates redundant characters found in JSON.

Objects are represented as key-value pairs:

name:John Doe
age:30
active:T

Arrays of objects with the same structure use a highly efficient tabular format:

users[3]{id,name,email}:
1|Alice|alice@example.com
2|Bob|bob@example.com
3|Charlie|charlie@example.com

This provides approximately 50% token savings compared to the equivalent JSON for tabular data.

Testing

This package supports PHP 7.0 through 8.4 and provides PHPUnit configurations for each version range.

# Run tests on PHP 8.1+ (PHPUnit 10+)
composer test

# Run tests on PHP 7.0-7.1 (PHPUnit 6-7)
composer test:php70

# Run tests on PHP 7.2-8.0 (PHPUnit 8-9)
composer test:php72

# Run tests with coverage
composer test:coverage

The test suite is compatible with:

PHPUnit Version	PHP Version	Config File
6.5.14 - 7.5.20	PHP 7.0 - 7.1	`phpunit-php70.xml.dist`
8.5.50 - 9.6.31	PHP 7.2 - 8.0	`phpunit-php72.xml.dist`
10.5.60+	PHP 8.1+	`phpunit.xml.dist`

Code Style

This package uses PHP-CS-Fixer for code style enforcement. Two configurations are provided for compatibility across PHP versions.

# Check code style (PHP 7.4+, php-cs-fixer 3.x)
composer cs-check

# Fix code style (PHP 7.4+, php-cs-fixer 3.x)
composer cs-fix

# Check code style (PHP 7.0-7.3, php-cs-fixer 2.x)
composer cs-check:v2

# Fix code style (PHP 7.0-7.3, php-cs-fixer 2.x)
composer cs-fix:v2

PHP-CS-Fixer Version	PHP Version	Config File
2.19.3	PHP 7.0 - 7.3	`.php-cs-fixer-v2.php`
3.x	PHP 7.4+	`.php-cs-fixer.dist.php`

PHP version compatibility

This library is designed to work across PHP 7.0 through 8.4:

PHP 7.0-7.4: Full support with polyfills for missing functions
PHP 8.0: Stringable interface support
PHP 8.1+: Enum support, readonly properties

Type support

TOON supports all PHP types that json_encode() supports:

PHP Type	TOON Representation
`null`	`N`
`bool`	`T` / `F`
`int`	Number literal
`float`	Number literal
`string`	Unquoted (with escapes)
`array` (sequential)	Pipe-separated values
`array` (associative)	Key-value pairs
`object`	Key-value pairs
`JsonSerializable`	Via `jsonSerialize()`
`DateTimeInterface`	ISO 8601 string
`Enum` (PHP 8.1+)	Backed value or name
`Stringable` (PHP 8.0+)	String conversion

Benchmarks

TOON vs JSON

Compared to JSON for typical API response data:

Metric	JSON	TOON	Improvement
Token Count	1000	600	40% fewer
Bytes	5000	3800	24% smaller

Token counts measured with OpenAI's tiktoken (cl100k_base)

Running Benchmarks

This library includes comprehensive benchmarks to measure performance:

# Compare against helgesverre/toon (requires both libraries installed)
composer benchmark:compare

# Run phpbench microbenchmarks
composer benchmark

# Run phpbench with summary report
composer benchmark:summary

# Run profiling benchmark
composer benchmark:profile

The comparative benchmark (composer benchmark:compare) will output:

Encoding/decoding times for small, medium, and large datasets
Operations per second
Memory usage
Speedup factors vs helgesverre/toon

Why is mfonte/fast-toon faster?

Key optimizations that make this library significantly faster:

Inline Type Checks: Direct type comparisons instead of function calls
Pre-computed Values: Indent strings, delimiter symbols cached at construction
Optimized String Building: Direct array appending instead of buffered writing
Fast Path Detection: Early returns for common primitive types
Minimal Memory Allocation: Reuse of internal buffers

Why Choose mfonte/fast-toon?

Consideration	mfonte/fast-toon	helgesverre/toon
PHP 7.0-7.4 support	✅ Yes	❌ No
PHP 8.0+ support	✅ Yes	✅ Yes
Encoding speed	⚡ 3x faster	baseline
Decoding speed	⚡ 15x faster	baseline
Memory efficiency	⚡ Optimized	standard
Spec compliance	✅ Full	✅ Full

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

License

The MIT License (MIT). Please see License File for more information.

Credits

Maurizio Fonte
TOON specification: github.com/toon-format

mfonte / fast-toon

Maintainers

Details