sterzik/tokenizer

Library for creating an universal tokenizer

Maintainers

Details

github.com/marek-sterzik/tokenizer

Homepage

Source

Issues

Installs: 1

Dependents: 1

Suggesters: 0

Security: 0

Stars: 0

Watchers: 0

Forks: 0

Open Issues: 0

pkg:composer/sterzik/tokenizer

dev-master 2025-10-14 22:57 UTC

Requires

  • php: >=8.1

MIT fc988e4e6afff8f6576f2dbc3c3171ff65916e41

  • Marek Sterzik <marek.sterzik.woop@gmail.com>

This package is auto-updated.

Last update: 2025-12-14 23:24:00 UTC

README

This library is currently in the very first stage of implementation and therefore not really useful.

But maybe in the future, there will be an univerzal tokenizer library.