mountsoftware/symfony-toon-serializer

Symfony Serializer integration for TOON (Token-Oriented Object Notation) on top of helgesverre/toon

v1.0.0 2025-11-19 17:27 UTC

Requires

• php: >=8.1
• helgesverre/toon: ^1.0
• symfony/serializer: ^6.4 || ^7.0

Requires (Dev)

• phpunit/phpunit: ^10.0 || ^11.0
• symfony/property-access: ^6.4 || ^7.0
• symfony/var-dumper: ^6.4 || ^7.0

MIT

• Mount Software

README

A Symfony Serializer integration for TOON (Token-Oriented Object Notation) format, built on top of the helgesverre/toon library.

TOON is a compact, human-readable data serialization format optimized for LLM contexts and token efficiency.

Installation

Install via Composer:

composer require mountsoftware/symfony-toon-serializer

Requirements

• PHP 8.1 or higher
• Symfony Serializer ^6.4 or ^7.0
• helgesverre/toon ^1.0

Features

• 🔌 Drop-in integration with Symfony Serializer component
• 📦 Format identifier: toon
• ⚙️ Configurable options: delimiters, strict mode, indentation
• ✅ Option validation: Early error detection with clear exceptions
• 🎨 Symfony conventions: Namespaced context options following best practices
• 🔒 Type-safe constants: Use ToonOptions for autocomplete & validation
• 🧪 Comprehensive test suite: 76 tests covering all scenarios
• 📝 Standalone service: Use TOON without the full serializer
• 🎯 Thin wrapper: Delegates all TOON logic to the proven helgesverre/toon library

Quick Start

Using with Symfony Serializer

use Symfony\Component\Serializer\Serializer;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;
use MountSoftware\SymfonyToonSerializer\Encoder\ToonEncoder;

$serializer = new Serializer(
    [new ObjectNormalizer()],
    [new ToonEncoder()]
);

// Serialize to TOON
$data = [
    'id' => 123,
    'name' => 'Alice',
    'active' => true,
];

$toon = $serializer->serialize($data, 'toon');
// Output:
// id: 123
// name: Alice
// active: true

// Deserialize from TOON
$decoded = $serializer->deserialize($toon, YourClass::class, 'toon');

Using the Standalone Service

use MountSoftware\SymfonyToonSerializer\ToonService;

$toonService = new ToonService();

// Encode
$toon = $toonService->encode([
    'user' => [
        'name' => 'Bob',
        'age' => 30,
    ],
]);

// Decode
$data = $toonService->decode($toon);

Configuration

Context Options

Following Symfony conventions, options must be namespaced under toon_options:

use MountSoftware\SymfonyToonSerializer\ToonOptions;

$toon = $serializer->serialize($data, 'toon', [
    'toon_options' => [
        ToonOptions::DELIMITER => ToonOptions::DELIMITER_TAB,
        ToonOptions::STRICT => true,
        ToonOptions::INDENT => 4,
    ],
]);

This prevents conflicts with other encoders and follows Symfony best practices.

Using Option Constants

The ToonOptions class provides constants for type safety:

use MountSoftware\SymfonyToonSerializer\ToonOptions;

// Option keys
ToonOptions::DELIMITER
ToonOptions::INDENT
ToonOptions::STRICT
ToonOptions::LENGTH_MARKER

// Delimiter values
ToonOptions::DELIMITER_COMMA  // ','
ToonOptions::DELIMITER_TAB    // "\t"
ToonOptions::DELIMITER_PIPE   // '|'

Available Options

Encoding Options

• delimiter (string, default: ','): Field delimiter for tabular data
  • Comma: ','
  • Tab: "\t"
  • Pipe: '|'
• indent (int, default: 2): Number of spaces for indentation
• lengthMarker (string|false, default: false): Prefix for array length markers (use '#' or false)

Decoding Options

• strict (bool, default: true): Enable strict mode validation
  • In strict mode: Validates array lengths, column counts, indentation
  • In lenient mode: More forgiving parsing
• indent (int, default: 2): Expected indentation level

Default Options

Set default options for the encoder:

$encoder = new ToonEncoder([
    'delimiter' => '|',
    'indent' => 4,
    'strict' => false,
]);

Context options override default options:

// Uses pipe delimiter (from defaults)
$toon1 = $encoder->encode($data, 'toon');

// Uses tab delimiter (from context)
$toon2 = $encoder->encode($data, 'toon', [
    'toon_options' => ['delimiter' => "\t"],
]);

Option Validation

The library validates all options and throws InvalidArgumentException for invalid values:

// Invalid delimiter - throws exception
$serializer->serialize($data, 'toon', [
    'toon_options' => [
        'delimiter' => ';',  // Only ',', "\t", '|' are valid
    ],
]);

// Invalid indent - throws exception
$serializer->serialize($data, 'toon', [
    'toon_options' => [
        'indent' => -1,  // Must be >= 0
    ],
]);

// Invalid strict mode - throws exception
$encoder->decode($toon, 'toon', [
    'toon_options' => [
        'strict' => 'true',  // Must be boolean, not string
    ],
]);

This prevents silent failures and catches configuration errors early.

Examples

Simple Object

$data = [
    'id' => 123,
    'name' => 'Ada',
    'active' => true,
];

$toon = $serializer->serialize($data, 'toon');

Output:

id: 123
name: Ada
active: true

Nested Object

$data = [
    'user' => [
        'id' => 123,
        'name' => 'Ada',
        'meta' => [
            'active' => true,
            'score' => 9.5,
        ],
    ],
];

Output:

user:
  id: 123
  name: Ada
  meta:
    active: true
    score: 9.5

Arrays of Primitives

$data = [
    'tags' => ['admin', 'ops', 'dev'],
];

Output:

tags[3]: admin,ops,dev

Tabular Data

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'role' => 'admin'],
        ['id' => 2, 'name' => 'Bob', 'role' => 'user'],
    ],
];

Output:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

Tab-Delimited Data

$data = [
    'items' => [
        ['sku' => 'A1', 'name' => 'Widget', 'qty' => 2],
        ['sku' => 'B2', 'name' => 'Gadget', 'qty' => 1],
    ],
];

$toon = $serializer->serialize($data, 'toon', [
    'toon_options' => ['delimiter' => "\t"],
]);

Output:

items[2	]{sku	name	qty}:
  A1	Widget	2
  B2	Gadget	1

Pipe-Delimited Data

$data = [
    'products' => [
        ['id' => 1, 'name' => 'Item A'],
        ['id' => 2, 'name' => 'Item B'],
    ],
];

$toon = $serializer->serialize($data, 'toon', [
    'toon_options' => ['delimiter' => '|'],
]);

Output:

products[2|]{id|name}:
  1|Item A
  2|Item B

Symfony Integration

Manual Service Registration

In config/services.yaml:

services:
  MountSoftware\SymfonyToonSerializer\Encoder\ToonEncoder:
    arguments:
      $defaultOptions:
        delimiter: ','
        strict: true
    tags:
      - { name: serializer.encoder }

  MountSoftware\SymfonyToonSerializer\ToonService:
    arguments:
      $defaultOptions:
        delimiter: ','

Auto-configuration

If using Symfony auto-configuration, the encoder will be automatically registered if tagged properly. The library is designed to work with Symfony's standard service discovery.

Testing

Run the test suite:

composer test

Or with PHPUnit directly:

vendor/bin/phpunit

The test suite includes 76 tests covering:

• ✅ Simple object encoding/decoding
• ✅ Nested object handling
• ✅ Primitive array inline notation
• ✅ Tabular array formats
• ✅ Mixed and heterogeneous arrays
• ✅ Delimiter variants (comma, tab, pipe)
• ✅ Quoting and special character handling
• ✅ Error handling and strict mode validation
• ✅ Empty objects and arrays
• ✅ Round-trip data integrity
• ✅ Option validation (delimiters, indent, strict mode)
• ✅ Context option extraction (namespaced)
• ✅ Invalid option error handling

TOON Format

TOON (Token-Oriented Object Notation) is a data serialization format designed for efficiency and readability. Key features:

• Compact: Optimized for minimal token usage in LLM contexts
• Human-readable: Easy to read and write by hand
• Structured: Supports objects, arrays, and primitives
• Tabular: Efficient representation of uniform data
• Flexible delimiters: Choose between comma, tab, or pipe

For more details on the TOON format, see the helgesverre/toon documentation.

Known Limitations

The underlying TOON library has some limitations for round-trip encoding/decoding:

1. Mixed arrays with nested objects: When encoding mixed arrays where list items contain objects (e.g., [1, ['a' => 1], 'text']), the nested object is decoded as a string rather than an associative array. This is due to how the TOON format represents inline objects in list contexts.

2. Arrays of arrays as list items: Similar to above, inline array notation in list items may not round-trip perfectly.

For best round-trip support, use:

• Tabular format for uniform arrays of objects
• Inline notation for simple primitive arrays
• Nested indentation for complex nested structures

Contributing

Contributions are welcome! Please ensure:

1. All tests pass: composer test
2. Code follows PSR-12 standards
3. New features include tests
4. Documentation is updated

License

MIT License. See LICENSE file for details.

Credits

• Built by Mount Software
• Uses helgesverre/toon for TOON encoding/decoding
• Integrates with Symfony Serializer

Links

• TOON Format Specification
• Symfony Serializer Documentation
• Issue Tracker