There is no license information available for the latest version (v2.0.1) of this package.

Math Tokenizer

v2.0.1 2020-04-14 12:13 UTC

This package is auto-updated.

Last update: 2020-05-27 09:58:29 UTC



Integrity check codecov License: MIT

Tokenizer is a simple library used to convert math formulas to arrays of tokens.

Please help to improve this documentation by sending a Pull request.

Install using Composer:

composer require mathematicator-core/tokenizer


Imagine you can:

  • Convert all your math formulas to a stream of tokens
  • Convert user math input to LaTeX
  • Solve your math problems using a calculator
  • Render the tokens tree map for debug

How to use

Inject the Tokenizer service through DIC and tokenize your query.

$tokenizer = new Tokenizer(/* some dependencies */);

// Convert math formula to an array of tokens:
$tokens = $tokenizer->tokenize('(5+3)*(2/(7+3))');

// Now you can convert tokens to a more useful format:
$objectTokens = $tokenizer->tokensToObject($tokens);

dump($objectTokens); // Return typed tokens with meta data

// Render to LaTeX
echo $tokenizer->tokensToLatex($objectTokens);

// Render to debug tree (extremely fast):
echo $tokenizer->renderTokensTree($objectTokens);


The tokenizer uses automatic configuration based on DIC. Just use the DIC container and the service will be fully available.


All new contributions should have its unit tests in /tests directory.

Before you send a PR, please, check all tests pass.

This package uses Nette Tester. You can run tests via command:

composer test

Before PR, please run complete code check via command:

composer cs:install # only first time
composer fix # otherwise pre-commit hook can fail