There is no license information available for the latest version (v1.0.5) of this package.

Math Tokenizer

v1.0.5 2020-01-23 11:08 UTC

This package is auto-updated.

Last update: 2020-02-20 08:39:34 UTC


Tokenizer is a simple library used to convert math formulas to arrays of tokens.

Please help to improve this documentation by sending a Pull request.

Install using Composer:

composer require mathematicator-core/tokenizer


Imagine you can:

  • Convert all your math formulas to a stream of tokens
  • Convert user math input to LaTeX
  • Solve your math problems using a calculator
  • Render the tokens tree map for debug

How to use

Inject the Tokenizer service through DIC and tokenize your query.

$tokenizer = new Tokenizer(/* some dependencies */);

// Convert math formula to an array of tokens:
$tokens = $tokenizer->tokenize('(5+3)*(2/(7+3))');

// Now you can convert tokens to a more useful format:
$objectTokens = $tokenizer->tokensToObject($tokens);

dump($objectTokens); // Return typed tokens with meta data

// Render to LaTeX
echo $tokenizer->tokensToLatex($objectTokens);

// Render to debug tree (extremely fast):
echo $tokenizer->renderTokensTree($objectTokens);


Tokenizer needs your math configuration.

In common.neon simply define the parameters:

            - sin
            - cos
            - tan
            - cotan
            - log
            - log\d*
            - ln
            - sqrt