prinsfrank / transliteration
A typed wrapper for the native PHP transliterator offering typed and documented arguments instead of an arbitrary string
Fund package maintenance!
PrinsFrank
Installs: 9 688
Dependents: 1
Suggesters: 0
Security: 0
Stars: 9
Watchers: 2
Forks: 1
Open Issues: 0
Requires
- php: ^8.1 || ^8.2 || ^8.3
- ext-intl: *
- prinsfrank/standards: ^3.6
Requires (Dev)
- friendsofphp/php-cs-fixer: ^3.46
- phpstan/phpstan: ^1.10
- phpstan/phpstan-strict-rules: ^1.5
- phpunit/phpunit: ^10.4
README
Transliteration
A typed wrapper for the native PHP transliterator offering typed and documented arguments instead of an arbitrary string
Setup
Note Make sure you are running PHP 8.1 or higher to use this package
To start right away, run the following command in your composer project;
composer require prinsfrank/transliteration
Or for development only;
composer require prinsfrank/transliteration --dev
Why this package?
Transliteration is to represent or spell in the characters of another alphabet. But Transliteration in PHP is so much more powerful than that. It's a shame that it is barely documented and that you'll have to look in the ICU documentation for a full feature list.
But worst of all: the arguments for transliterator creation are strings. This package provides a full strictly-typed wrapper to create transliterators. It also provides some ready-to-go ConversionSets to get you up and running in no time!
Use Cases
Personal communication
Let's say you have a platform (shop, SAAS etc.) in many languages, but Customer Support staff speaks only a few of them. They have the platform available in their local language(s), but you can't translate things like names or addresses.
So a customer calls, and they have put in their name as 이준
. How do you pronounce this as someone that doesn't speak Korean but only English? Simple: You transliterate it and display it next to their name: 이준 (Ijun)
Identity verification
Another example: According to local laws and regulations you have to verify the identity of users on your platform (Banks, Employers etc.). They have entered their official name as Ivan
, but their identity document says their name is Иван
. As a person that doesn't read Cyrillic this might be hard to tell, but these in fact are the same. It might make sense here to display a name in multiple scripts depending on the country of the identity document.
Transliteration basics
The transliteratorBuilder is the main entrypoint in this package. It provides a Generic but strictly-typed interface to the ext-intl transliterator.
Simple Conversions
The easiest example might be to replace one letter with some others, for example in German:
(new TransliteratorBuilder()) ->addConversion(new Conversion('ß', 'ss')) ->transliterate('Straße'); // Strasse
Behind the scenes, this does eventually get converted to a string that is used to construct a rule based ext-intl transliterator; 'ß>ss;'
In this example it's not clear where this package shines, so let's move on to something more challenging.
Script conversions
Let's say we have a bunch of texts in arbitrary scripts, and we want to make sure that they can all be represented in ASCII.
(new TransliteratorBuilder()) ->applyConversionSet(new ToASCII()) ->transliterate('アマゾン'); // amazon
As there is no direct transliteration from Any to ASCII possible, this results in the following conversion behind the scenes: 'Any-Latin;Latin-ASCII;'
IPA to English approximation
Now we're going one step further: Somewhere along the line one of our conversions results in an IPA character instead of English one. What if we could also approximate that in English? This package has you covered!
(new TransliteratorBuilder()) ->applyConversionSet(new ToASCII()) ->applyConversionSet(new IPAToEnglishApproximation()) ->transliterate('naɕi gʌba'); // naci guba
Behind the scenes the following ruleSet string gets created for you:
::Any-Latin;::Latin-ASCII;dʒ>g;kʰ>c;kʷ>qu;kᶣ>cu;ɫ>ll;ŋ>n;Ŋ>N;ɲ>n;Ɲ>N;pʰ>p;ʃ>sh;Ʃ>SH;tʰ>t;tʃ>ch;aː>a;Aː>A;ɛ>e;Ɛ>E;eː>a;Eː>A;ɪ>i;Ɪ>I;iː>i;Iː>I;ɔ>o;Ɔ>O;oː>aw;ʊ>u;Ʊ>U;ʌ>u;Ʌ>U;uː>u;yː>u;ae̯>igh;oe̯>oy;au̯>ow;ei̯>ay;ui̯>ui;
Let's ignore the resulting ruleSet strings for the rest of this documentation, as the goal of this package is to provide you an abstract but typed interface for the transliteration package, without requiring you to understand the underlying syntax.
Conversion sets
Conversion sets are the biggest building blocks in this package. They can be applied by calling the applyConversionSet
method for each set;
(new TransliteratorBuilder()) ->applyConversionSet(new ToASCII()) ->applyConversionSet(new IPAToEnglishApproximation())
Or calling the applyConversionSets
method with an array of conversion sets;
(new TransliteratorBuilder()) ->applyConversionSets([ new ToASCII(), new IPAToEnglishApproximation(), ])
Bundled conversion sets
There are a bunch of conversion sets included in this package:
Creating a custom conversion set
To create a custom conversion set, simply
- Add the ConversionSet interface to your class;
use PrinsFrank\Transliteration\ConversionSet; class CustomConversionSet implements ConversionSet { }
- Add an implementation for the apply method;
public function apply(TransliteratorBuilder $transliteratorBuilder): void { // Add your code here }
- And pass your new custom conversion to the transliteratorBuilder;
(new TransliteratorBuilder()) ->applyConversionSet(new CustomConversionSet());
If the ConversionSet needs any arguments, you can implement a constructor and use those arguments in the apply method. For a simple example of this, see the Replace ConversionSet.
SingleID
A SingleID is the most basic conversion available. It expects a BasicID and an optional filter. The arguments of the BasicID are self documenting; Either pass a ScriptName, ScriptAlias, Language or SpecialTag to the target (and optionally the source) to convert to (and from) scripts and languages or special tags;
public function apply(TransliteratorBuilder $transliteratorBuilder): void { $transliteratorBuilder->addSingleID( new SingleID( new BasicID(SpecialTag::ASCII, SpecialTag::Any), ) ); }
Conversion
To add custom conversions from A to B, it is possible to add those directly;
public function apply(TransliteratorBuilder $transliteratorBuilder): void { $transliteratorBuilder->addConversion( new Conversion('A', 'B') ); }
There are some extra parameters available here: beforeContext
, afterContext
and resultToRevisit
. Those are currently undocumented here, but can be thoroughly read about in the ICU rule documentation.
VariableDefinition
Variables can be created to reuse them later. For example, if you want to reuse the string 'bar' a lot, you can create a named variable:
public function apply(TransliteratorBuilder $transliteratorBuilder): void { $transliteratorBuilder->addVariableDefinition( new VariableDefinition('foo', 'bar') ); }
Please note that variable names cannot be empty and cannot contain any special characters. If you do either of these the constructor of VariableDefinition will throw an exception warning you of such. Variable values will be automatically escaped if they contain any special characters.
(Global) filters
Global filters can technically be modified from the scope of a ConversionSet, but that should not be done. If ConversionSet A modifies the global filter, and then ConversionSet B modifies it, ConversionSet A will not function properly anymore. Instead, make sure to add a filter on SingleID level for each ID;
public function apply(TransliteratorBuilder $transliteratorBuilder): void { $transliteratorBuilder->addSingleID( new SingleID( new BasicID(SpecialTag::ASCII, SpecialTag::Any), (new Filter())->addRange(new Character('a'), new Character('z')) ) ); }
ConversionSet nesting
It is possible to nest conversion sets;
public function apply(TransliteratorBuilder $transliteratorBuilder): void { $transliteratorBuilder->applyConversionSet(new ToASCII()); }
To prevent recursion it is not possible to use a ConversionSet that is already called in a ConversionSet chain. For example: it is possible to use A in B and B in C, but not A in B and B in A.
Technical: Transliterator composition
To give users of this package a nice and readable argument list, the formalIDSyntax to create a normal transliterator and the TransformRuleSyntax have both been implemented in this package according to the specification. Their relations are as follows:
When the transliterator is 'built' by calling applyConversionSet
, applyConversionSets
, addSingleID
, addConversion
or AddVariableDefinition
, the internal array $conversions
is filled either directly, or by the conversionSet in the apply
method.
When subsequently creating the transliterator - either by calling getTransliterator
or transliterate
- this package checks the simplest type of transliterator that can be used:
RuleLists and CompoundID are not directly constructed, as you can't know with fluid notation if they will eventually be needed. For example: when adding a SingleID, and subsequently adding another SingleID, we should convert everything to a CompoundID. We can't know that while adding our first SingleID though. That's why the CompoundID and RuleList are virtual internal classes that are applied as wrappers at runtime (and hence marked as internal). The logic for this lives in the transliterate
, getTransliterator
and containsRuleSyntax
methods in the TranliteratorBuilder itself.