Search by

sterzik / tokenizer

marek.sterzik

Library for creating an universal tokenizer

Package info

github.com/marek-sterzik/tokenizer

pkg:composer/sterzik/tokenizer

Security

Advisories: 0

Aikido package health analysis

Transparency log

Statistics

Installs: 1

Dependents: 1

Suggesters: 0

Stars: 0

Open Issues: 0

dev-master 2025-10-14 22:57 UTC

Requires

  • php: >=8.1

Requires (Dev)

None

Suggests

None

Provides

None

Conflicts

None

Replaces

None

MIT fc988e4e6afff8f6576f2dbc3c3171ff65916e41

  • Marek Sterzik <marek.sterzik.woop@gmail.com>

Published on Packagist.org on 2025-10-14 22:59 UTC

This package is auto-updated.

Last update: 2026-08-15 00:51:27 UTC

README

This library is currently in the very first stage of implementation and therefore not really useful.

But maybe in the future, there will be an univerzal tokenizer library.