README

A Laravel package for TOON (Token-Oriented Object Notation) format with comprehensive database and LLM support. TOON is a compact, human-readable encoding of the JSON data model that minimizes tokens and makes structure easy for models to follow.

What is TOON?

TOON combines YAML's indentation-based structure for nested objects with a CSV-style tabular layout for uniform arrays. It's optimized for LLM input as a drop-in, lossless representation of your existing JSON data.

Example:

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

Becomes:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

This reduces token count significantly while maintaining explicit structure that helps LLMs parse and validate data reliably.

Another Example:

Standard JSON:

{
  "products": [
    { "sku": "ABC123", "name": "Widget", "price": 29.99, "stock": 150 },
    { "sku": "XYZ789", "name": "Gadget", "price": 49.99, "stock": 75 },
    { "sku": "DEF456", "name": "Thing", "price": 19.99, "stock": 200 }
  ]
}

TOON format:

products[3]{name,price,sku,stock}:
  Widget,29.99,ABC123,150
  Gadget,49.99,XYZ789,75
  Thing,19.99,DEF456,200

The [3] declares the array length, {name,price,sku,stock} declares the field names once, and each row is a compact comma-separated list of values. This approach declares structure once and streams data compactly, approaching CSV's efficiency while adding explicit structure.

Requirements

PHP 8.4 or higher
Laravel 12.0 or higher

Installation

Install the package via Composer:

composer require laravelplus/toon

The package will automatically register its service provider and facade.

Configuration

Publish the configuration file (optional):

php artisan vendor:publish --tag=toon-config

This will create config/toon.php with the following options:

return [
    'encoding' => [
        'tabular_threshold' => 2,  // Minimum items for tabular format
        'max_depth' => 100,        // Maximum nesting depth
    ],
    'llm' => [
        'validate_array_lengths' => true,   // Validate [N] declarations
        'validate_field_counts' => true,    // Validate field counts
    ],
];

Usage

Basic Encoding and Decoding

Using Facade

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'role' => 'admin'],
        ['id' => 2, 'name' => 'Bob', 'role' => 'user'],
    ],
];

// Encode to TOON format
$toon = Toon::encode($data);

// Decode back to array
$decoded = Toon::decode($toon);

Using Global Helper Function

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'role' => 'admin'],
        ['id' => 2, 'name' => 'Bob', 'role' => 'user'],
    ],
];

// Fluent interface - pass data to toon()
$toon = toon($data)->encode();

// Or encode directly
$toon = toon()->encode($data);

// Decode
$decoded = toon($toon)->decode();

// Or decode directly
$decoded = toon()->decode($toon);

Output:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

Database Support

Eloquent Models

use LaravelPlus\Toon\Facades\Toon;
use App\Models\User;

// Single model
$user = User::find(1);
$toon = Toon::fromEloquent($user);

// With relationships
$toon = Toon::fromEloquent($user, ['posts', 'comments']);

// Collection
$users = User::all();
$toon = Toon::fromEloquentCollection($users);

// Collection with relationships
$toon = Toon::fromEloquentCollection($users, ['profile']);

Using Helper Function

use App\Models\User;

// Fluent interface
$user = User::find(1);
$toon = toon($user)->fromEloquent();

// With relationships
$toon = toon($user)->fromEloquent(['posts', 'comments']);

// Collection
$users = User::all();
$toon = toon($users)->fromEloquentCollection();

// Or without passing to toon()
$toon = toon()->fromEloquent($user);

Query Builder

use LaravelPlus\Toon\Facades\Toon;

// Basic query
$toon = Toon::fromQueryBuilder(
    User::query()->where('active', true)
);

// With specific columns
$toon = Toon::fromQueryBuilder(
    User::query(),
    ['id', 'name', 'email']
);

// Chunked processing for large datasets
use LaravelPlus\Toon\Database\QueryBuilderToonConverter;

foreach (QueryBuilderToonConverter::toToonChunked(
    User::query(),
    chunkSize: 1000
) as $chunkToon) {
    // Process each chunk
    echo $chunkToon;
}

Collections

use LaravelPlus\Toon\Facades\Toon;
use Illuminate\Support\Collection;

$collection = collect([
    ['id' => 1, 'name' => 'Alice'],
    ['id' => 2, 'name' => 'Bob'],
]);

$toon = Toon::fromCollection($collection);

LLM Support

Encoding for LLM Prompts

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'score' => 95],
        ['id' => 2, 'name' => 'Bob', 'score' => 87],
    ],
];

// Encode with validation metadata
$toon = Toon::forLlm($data);

// Validate TOON string
$validation = Toon::validate($toon);
if ($validation['valid']) {
    // Use the TOON string
} else {
    foreach ($validation['errors'] as $error) {
        echo $error;
    }
}

Decoding LLM Responses

use LaravelPlus\Toon\Facades\Toon;

// Basic decoding
$llmResponse = "users[2]{id,name}:\n  1,Alice\n  2,Bob";
$data = Toon::fromLlm($llmResponse);

// Decode with validation
$result = Toon::fromLlmWithValidation($llmResponse);

if ($result['valid']) {
    $data = $result['data'];
} else {
    foreach ($result['errors'] as $error) {
        echo "Error: {$error}\n";
    }
}

Building LLM Prompts

use LaravelPlus\Toon\Facades\Toon;

// Custom prompt
$prompt = Toon::prompt()
    ->system('You are a data analyst.')
    ->user('Analyze the user data and identify trends.')
    ->withData([
        'users' => User::all()->toArray(),
    ])
    ->build();

// Pre-built prompt types
$analysisPrompt = Toon::prompt()->forAnalysis(
    'What trends do you see in the data?',
    ['users' => User::all()->toArray()]
);

$transformationPrompt = Toon::prompt()->forTransformation(
    'Convert all names to uppercase',
    ['users' => User::all()->toArray()]
);

$extractionPrompt = Toon::prompt()->forExtraction(
    'Extract all email addresses',
    ['users' => User::all()->toArray()]
);

Format Conversion

TOON supports conversion to and from multiple formats: JSON, XML, YAML, CSV, and XLSX.

TOON to JSON and Back

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice'],
        ['id' => 2, 'name' => 'Bob'],
    ],
];

// Encode to TOON
$toon = Toon::encode($data);

// Convert TOON to JSON
$json = Toon::toJson($toon);

// Convert JSON back to TOON
$toonFromJson = Toon::fromJson($json);

// Using helper function
$json = toon($toon)->toJson();
$toon = toon($json)->fromJson();

TOON to XML and Back

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice'],
        ['id' => 2, 'name' => 'Bob'],
    ],
];

// Encode to TOON
$toon = Toon::encode($data);

// Convert TOON to XML
$xml = Toon::toXml($toon, rootElement: 'data');

// Convert XML back to TOON
$toonFromXml = Toon::fromXml($xml);

// Using helper function
$xml = toon($toon)->toXml(rootElement: 'data');
$toon = toon($xml)->fromXml();

TOON to YAML and Back

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice'],
        ['id' => 2, 'name' => 'Bob'],
    ],
];

// Encode to TOON
$toon = Toon::encode($data);

// Convert TOON to YAML
$yaml = Toon::toYaml($toon, inline: 2, indent: 2);

// Convert YAML back to TOON
$toonFromYaml = Toon::fromYaml($yaml);

// Using helper function
$yaml = toon($toon)->toYaml();
$toon = toon($yaml)->fromYaml();

TOON to CSV and Back

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'role' => 'admin'],
        ['id' => 2, 'name' => 'Bob', 'role' => 'user'],
    ],
];

// Encode to TOON
$toon = Toon::encode($data);

// Convert TOON to CSV
$csv = Toon::toCsv($toon, delimiter: ',', includeHeaders: true);

// Convert CSV back to TOON
$toonFromCsv = Toon::fromCsv($csv);

// Using helper function
$csv = toon($toon)->toCsv();
$toon = toon($csv)->fromCsv();

// Custom CSV options
$csv = Toon::toCsv($toon, delimiter: ';', enclosure: "'", includeHeaders: false);

TOON to XLSX and Back

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'role' => 'admin'],
        ['id' => 2, 'name' => 'Bob', 'role' => 'user'],
    ],
];

// Encode to TOON
$toon = Toon::encode($data);

// Convert TOON to XLSX (returns base64 encoded string)
$xlsxBase64 = Toon::toXlsx($toon, includeHeaders: true, sheetName: 'Users');

// Save to file
file_put_contents('users.xlsx', base64_decode($xlsxBase64));

// Convert XLSX file back to TOON
$toonFromXlsx = Toon::fromXlsx('users.xlsx', sheetName: 'Users');

// Or use base64 encoded content
$toonFromXlsx = Toon::fromXlsx($xlsxBase64);

// Using helper function
$xlsx = toon($toon)->toXlsx(sheetName: 'Data');
$toon = toon('file.xlsx')->fromXlsx();

Direct Format Conversion

use LaravelPlus\Toon\Converters\JsonConverter;
use LaravelPlus\Toon\Converters\XmlConverter;
use LaravelPlus\Toon\Converters\YamlConverter;
use LaravelPlus\Toon\Converters\CsvConverter;
use LaravelPlus\Toon\Converters\XlsxConverter;

// JSON to TOON
$json = '{"users":[{"id":1,"name":"Alice"}]}';
$toon = JsonConverter::fromJson($json);

// TOON to JSON
$json = JsonConverter::toJson($toon);

// XML to TOON
$xml = '<?xml version="1.0"?><root><users><id>1</id><name>Alice</name></users></root>';
$toon = XmlConverter::fromXml($xml);

// TOON to XML
$xml = XmlConverter::toXml($toon, rootElement: 'data');

// YAML to TOON
$yaml = "users:\n  - id: 1\n    name: Alice";
$toon = YamlConverter::fromYaml($yaml);

// TOON to YAML
$yaml = YamlConverter::toYaml($toon, inline: 2, indent: 2);

// CSV to TOON
$csv = "id,name,role\n1,Alice,admin\n2,Bob,user";
$toon = CsvConverter::fromCsv($csv);

// TOON to CSV
$csv = CsvConverter::toCsv($toon, includeHeaders: true);

// XLSX to TOON
$toon = XlsxConverter::fromXlsx('data.xlsx', sheetName: 'Sheet1');

// TOON to XLSX (returns base64 encoded string)
$xlsxBase64 = XlsxConverter::toXlsx($toon, includeHeaders: true, sheetName: 'Data');

Advanced Examples

Complex Nested Data

use LaravelPlus\Toon\Facades\Toon;

$data = [
    'context' => [
        'task' => 'Our favorite hikes together',
        'location' => 'Boulder',
        'season' => 'spring_2025',
    ],
    'friends' => ['ana', 'luis', 'sam'],
    'hikes' => [
        [
            'id' => 1,
            'name' => 'Blue Lake Trail',
            'distanceKm' => 7.5,
            'elevationGain' => 320,
            'companion' => 'ana',
            'wasSunny' => true,
        ],
        [
            'id' => 2,
            'name' => 'Ridge Overlook',
            'distanceKm' => 9.2,
            'elevationGain' => 540,
            'companion' => 'luis',
            'wasSunny' => false,
        ],
    ],
];

$toon = Toon::encode($data);

Output:

context:
  task: Our favorite hikes together
  location: Boulder
  season: spring_2025
friends[3]: ana,luis,sam
hikes[2]{companion,distanceKm,elevationGain,id,name,wasSunny}:
  1,Blue Lake Trail,7.5,320,ana,true
  2,Ridge Overlook,9.2,540,luis,false

Using Without Facade

use LaravelPlus\Toon\ToonManager;

$manager = app(ToonManager::class);
$toon = $manager->encode($data);

Using Global Helper Function

The toon() helper function provides a fluent interface for all TOON operations:

// Encode - fluent style
$toon = toon($data)->encode();

// Encode - direct style
$toon = toon()->encode($data);

// Decode - fluent style
$data = toon($toon)->decode();

// Decode - direct style
$data = toon()->decode($toon);

// Database operations
$toon = toon($user)->fromEloquent();
$toon = toon($users)->fromEloquentCollection(['profile']);
$toon = toon($query)->fromQueryBuilder();
$toon = toon($collection)->fromCollection();

// LLM operations
$toon = toon($data)->forLlm();
$data = toon($toon)->fromLlm();
$result = toon($toon)->fromLlmWithValidation();

// Validation
$validation = toon($toon)->validate();

// Format conversion
$json = toon($toon)->toJson();
$toon = toon($json)->fromJson();
$xml = toon($toon)->toXml();
$toon = toon($xml)->fromXml();
$yaml = toon($toon)->toYaml();
$toon = toon($yaml)->fromYaml();
$csv = toon($toon)->toCsv();
$toon = toon($csv)->fromCsv();
$xlsx = toon($toon)->toXlsx();
$toon = toon($xlsx)->fromXlsx();

// Prompt builder
$prompt = toon()->prompt()
    ->system('You are helpful')
    ->user('Analyze this')
    ->withData($data)
    ->build();

Architecture

This package follows SOLID principles with interfaces and abstractions:

Interfaces

ToonManagerInterface - Main interface for TOON operations
LlmEncoderInterface - Interface for LLM-specific encoding
LlmDecoderInterface - Interface for LLM response decoding
PromptBuilderInterface - Interface for building LLM prompts
DatabaseConverterInterface - Interface for database converters

Abstract Classes

AbstractToonConverter - Base class for custom TOON converters

Dependency Injection

You can type-hint interfaces in your classes:

use LaravelPlus\Toon\Contracts\ToonManagerInterface;

class MyService
{
    public function __construct(
        private ToonManagerInterface $toon
    ) {}

    public function processData(array $data): string
    {
        return $this->toon->encode($data);
    }
}

API Reference

Facade Methods

`Toon::encode(array $data): string`

Encode an array to TOON format.

`Toon::decode(string $toon): array`

Decode TOON format to an array.

`Toon::fromEloquent(Model $model, ?array $relations = null): string`

Convert an Eloquent model to TOON format.

`Toon::fromEloquentCollection(EloquentCollection $collection, ?array $relations = null): string`

Convert an Eloquent collection to TOON format.

`Toon::fromQueryBuilder(Builder|QueryBuilder $query, ?array $columns = null): string`

Convert Query Builder results to TOON format.

`Toon::fromCollection(Collection $collection): string`

Convert a Laravel Collection to TOON format.

`Toon::forLlm(array $data): string`

Encode data specifically for LLM prompts.

`Toon::fromLlm(string $toon): array`

Decode LLM response from TOON format.

`Toon::fromLlmWithValidation(string $toon): array`

Decode LLM response with validation. Returns ['data' => array, 'valid' => bool, 'errors' => array].

`Toon::validate(string $toon): array`

Validate a TOON string. Returns ['valid' => bool, 'errors' => array].

`Toon::prompt(): PromptBuilder`

Create a new prompt builder instance.

`Toon::toJson(string $toon, int $flags = JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE): string`

Convert TOON format to JSON.

`Toon::fromJson(string $json, int $depth = 512): string`

Convert JSON to TOON format.

`Toon::toXml(string $toon, string $rootElement = 'root', string $version = '1.0', string $encoding = 'UTF-8'): string`

Convert TOON format to XML.

`Toon::fromXml(string $xml): string`

Convert XML to TOON format.

`Toon::toYaml(string $toon, int $inline = 2, int $indent = 2, int $flags = 0): string`

Convert TOON format to YAML.

`Toon::fromYaml(string $yaml, int $flags = 0): string`

Convert YAML to TOON format.

`Toon::toCsv(string $toon, string $delimiter = ',', string $enclosure = '"', string $escape = '\\', bool $includeHeaders = true): string`

Convert TOON format to CSV.

`Toon::fromCsv(string $csv, string $delimiter = ',', string $enclosure = '"', string $escape = '\\'): string`

Convert CSV to TOON format.

`Toon::toXlsx(string $toon, bool $includeHeaders = true, string $sheetName = 'Sheet1'): string`

Convert TOON format to XLSX. Returns base64 encoded XLSX content.

`Toon::fromXlsx(string $xlsxPath, ?string $sheetName = null): string`

Convert XLSX file (path or base64 encoded content) to TOON format.

PromptBuilder Methods

`system(string $prompt): self`

Set the system prompt.

`user(string $prompt): self`

Set the user prompt.

`withData(array $data): self`

Add data to include in TOON format.

`withInstructions(bool $include = true): self`

Include TOON format instructions in the prompt.

`build(): string`

Build and return the complete prompt.

`forAnalysis(string $question, array $data): string`

Build a prompt for data analysis.

`forTransformation(string $instructions, array $data): string`

Build a prompt for data transformation.

`forExtraction(string $question, array $data): string`

Build a prompt for data extraction.

Testing

Run the test suite using Pest:

./vendor/bin/pest

Or with PHPUnit:

./vendor/bin/phpunit

Code Quality

This package uses:

Pest PHP for testing
Rector for automated refactoring

Run Rector to analyze and fix code:

./vendor/bin/rector process src --dry-run
./vendor/bin/rector process src

TOON Format Specification

TOON supports three main formats:

Tabular Arrays: For uniform arrays of objects

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

Inline Arrays: For arrays of primitives
```
tags[3]: php,laravel,toon
```

Objects: For nested structures

user:
  name: Alice
  profile:
    bio: Developer

Performance

TOON format is optimized for:

Token efficiency: Reduces token count by 50-70% for uniform arrays
LLM parsing: Explicit structure helps models parse reliably
Validation: Built-in array length and field count validation

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This package is open-sourced software licensed under the MIT license.

Credits

TOON format specification: toonformat.dev
LaravelPlus Team

Support

For issues, questions, or contributions, please visit the GitHub repository.

laravelplus / toon

Maintainers

Details

README

What is TOON?

Requirements

Installation

Configuration

Usage

Basic Encoding and Decoding

Using Facade

Using Global Helper Function

Database Support

Eloquent Models

Using Helper Function

Query Builder

Collections

LLM Support

Encoding for LLM Prompts

Decoding LLM Responses

Building LLM Prompts

Format Conversion

TOON to JSON and Back

TOON to XML and Back

TOON to YAML and Back

TOON to CSV and Back

TOON to XLSX and Back

Direct Format Conversion

Advanced Examples

Complex Nested Data

Using Without Facade

Using Global Helper Function

Architecture

Interfaces

Abstract Classes

Dependency Injection

API Reference

Facade Methods

Toon::encode(array $data): string

Toon::decode(string $toon): array

Toon::fromEloquent(Model $model, ?array $relations = null): string

Toon::fromEloquentCollection(EloquentCollection $collection, ?array $relations = null): string

Toon::fromQueryBuilder(Builder|QueryBuilder $query, ?array $columns = null): string

Toon::fromCollection(Collection $collection): string

Toon::forLlm(array $data): string

Toon::fromLlm(string $toon): array

Toon::fromLlmWithValidation(string $toon): array

Toon::validate(string $toon): array

Toon::prompt(): PromptBuilder

Toon::toJson(string $toon, int $flags = JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE): string

Toon::fromJson(string $json, int $depth = 512): string

Toon::toXml(string $toon, string $rootElement = 'root', string $version = '1.0', string $encoding = 'UTF-8'): string

Toon::fromXml(string $xml): string

Toon::toYaml(string $toon, int $inline = 2, int $indent = 2, int $flags = 0): string

Toon::fromYaml(string $yaml, int $flags = 0): string

Toon::toCsv(string $toon, string $delimiter = ',', string $enclosure = '"', string $escape = '\\', bool $includeHeaders = true): string

Toon::fromCsv(string $csv, string $delimiter = ',', string $enclosure = '"', string $escape = '\\'): string

Toon::toXlsx(string $toon, bool $includeHeaders = true, string $sheetName = 'Sheet1'): string

Toon::fromXlsx(string $xlsxPath, ?string $sheetName = null): string

PromptBuilder Methods

system(string $prompt): self

user(string $prompt): self

withData(array $data): self

withInstructions(bool $include = true): self

build(): string

forAnalysis(string $question, array $data): string

forTransformation(string $instructions, array $data): string

forExtraction(string $question, array $data): string

Testing

Code Quality

TOON Format Specification

Performance

Contributing

License

Credits

Support

`Toon::encode(array $data): string`

`Toon::decode(string $toon): array`

`Toon::fromEloquent(Model $model, ?array $relations = null): string`

`Toon::fromEloquentCollection(EloquentCollection $collection, ?array $relations = null): string`

`Toon::fromQueryBuilder(Builder|QueryBuilder $query, ?array $columns = null): string`

`Toon::fromCollection(Collection $collection): string`

`Toon::forLlm(array $data): string`

`Toon::fromLlm(string $toon): array`

`Toon::fromLlmWithValidation(string $toon): array`

`Toon::validate(string $toon): array`

`Toon::prompt(): PromptBuilder`

`Toon::toJson(string $toon, int $flags = JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE): string`

`Toon::fromJson(string $json, int $depth = 512): string`

`Toon::toXml(string $toon, string $rootElement = 'root', string $version = '1.0', string $encoding = 'UTF-8'): string`

`Toon::fromXml(string $xml): string`

`Toon::toYaml(string $toon, int $inline = 2, int $indent = 2, int $flags = 0): string`

`Toon::fromYaml(string $yaml, int $flags = 0): string`

`Toon::toCsv(string $toon, string $delimiter = ',', string $enclosure = '"', string $escape = '\\', bool $includeHeaders = true): string`

`Toon::fromCsv(string $csv, string $delimiter = ',', string $enclosure = '"', string $escape = '\\'): string`

`Toon::toXlsx(string $toon, bool $includeHeaders = true, string $sheetName = 'Sheet1'): string`

`Toon::fromXlsx(string $xlsxPath, ?string $sheetName = null): string`

`system(string $prompt): self`

`user(string $prompt): self`

`withData(array $data): self`

`withInstructions(bool $include = true): self`

`build(): string`

`forAnalysis(string $question, array $data): string`

`forTransformation(string $instructions, array $data): string`

`forExtraction(string $question, array $data): string`