componenta/tokenizer

Token-based PHP source inspection helpers for Componenta

v1.0.0 2026-06-15 11:07 UTC

Requires

• php: ^8.4

MIT

README

PHP declaration tokenizer used by class discovery.

Installation

composer require componenta/tokenizer

Requirements

• PHP 8.4+

Related Packages

Package	Why it matters here
componenta/class-finder	Walks PHP files and uses this tokenizer for each source file.
componenta/app and *-app packages	Use class discovery during application cache compilation.
PHP Reflection	Used only when ClassInfo::$reflector is requested and the declaration is autoloadable.

What It Provides

• TokenizerInterface: public contract for parsing PHP source.
• Tokenizer: token-based parser for named declarations.
• ClassInfo: immutable declaration metadata with lazy reflection.
• DeclarationType: enum for class, interface, trait, and enum declarations.

Anonymous classes are ignored. The tokenizer assumes syntactically valid PHP input.

Parsing Source

use Componenta\Tokenizer\Tokenizer;

$declarations = (new Tokenizer())->parse(<<<'PHP'
<?php

namespace App;

final readonly class User {}
interface UserRepository {}
trait Timestamped {}
enum UserStatus: string { case Active = 'active'; }
PHP);

Each item is a ClassInfo:

$info = $declarations[0];

$info->fullyQualifiedName; // App\User
$info->name;               // User
$info->namespace;          // App
$info->type;               // DeclarationType::Class_
$info->isFinal;            // true
$info->isReadonly;         // true
$info->isConcrete;         // true

Search Modes

Use bitmasks to restrict declaration types:

use Componenta\Tokenizer\TokenizerInterface;

$classes = $tokenizer->parse($source, TokenizerInterface::SEARCH_CLASSES);
$contracts = $tokenizer->parse(
    $source,
    TokenizerInterface::SEARCH_INTERFACES | TokenizerInterface::SEARCH_TRAITS,
);

Available flags:

• SEARCH_CLASSES
• SEARCH_INTERFACES
• SEARCH_TRAITS
• SEARCH_ENUMS
• SEARCH_ALL

Reflection

ClassInfo::$reflector is lazy and requires the declaration to already be loaded.

if ($info->exists()) {
    $reflector = $info->reflector;
}