cageis/lexer

Used to separate the characters of a string into tokens.

1.0.0 2019-06-19 23:54 UTC

Requires

• php: ^7.3.0

Requires (Dev)

• phpunit/phpunit: ^7

MIT

• Jeffrey Lemay

README

Lexer is a minimal string parser that can be used to convert text into a list of tokens.

Example Usage

<?php
$lexer = CageIs\Lexer\LexerFactory::create();
$tokens = $lexer->addTokenPattern(new CageIs\Lexer\TokenPattern('\d+', 'num'))
    ->addTokenPattern(new CageIs\Lexer\TokenPattern('[a-zA-Z]+', 'alpha'))
    ->setWhitespaceIgnore(true)
    ->parse("hello\n123\n^&111111")
    ->toArray();

// The results will be
[
  [
    'name' => "alpha",
    'match' => "hello",
  ],
  [
    'name' => "num",
    'match' => "123",
  ],
  [
    'name' => "Unknown",
    'match' => "^",
  ],
  [
    'name' => "Unknown",
    'match' => "&",
  ],
  [
    'name' => "num",
    'match' => "111111",
  ]
];

Note: not calling $tokens->toArray() will return a collection object (CageIs\Lexer\TokenizedCollection) with a couple of helpful collection methods.