samt / nass
Arabic text normalization and diacritic-insensitive search for Quran, hadith and heritage texts, with match offsets in the original vocalized text.
Fund package maintenance!
Requires
- php: >=8.3
- ext-mbstring: *
Requires (Dev)
None
Suggests
None
Provides
None
Conflicts
None
Replaces
None
README
nass · نَصّ
Arabic text normalization and diacritic-insensitive search for Quran, hadith and heritage texts —
with match offsets in the original vocalized text.
One specification · four byte-identical packages · Dart, TypeScript, PHP and Rust
العربية · Specification · Test vectors · Corpus & isnad accuracy
search «الاعمال»
in إِنَّمَا الْأَعْمَالُ بِالنِّيَّاتِ
→ إِنَّمَا <mark>الْأَعْمَالُ</mark> بِالنِّيَّاتِ (start 9, end 21 in the original)
Contents
- Why nass
- Packages
- Quick start
- Normalization levels
- splitIsnad (experimental)
- Quality and testing
- Performance
- Compared with other libraries
- Repository layout
- Contributing
- License
Why nass
Searching vocalized Arabic is easy to get almost right. Strip the diacritics and you can find the word, but you lose its position in the text the reader sees. Quranic marks, presentation forms such as «ﷲ» and «ﻻ», honorific ligatures such as «ﷺ», invisible direction marks and Persian letter variants each break naive search in their own way. Most libraries cover a subset, in one language, with no shared tests.
| Offsets in the original | Every normalized character maps back to its source span, so you highlight the vocalized text the user is actually reading. |
| One spec, four languages | Dart, TypeScript, PHP and Rust pass the same 1,554 shared test vectors, plus a differential test against an independent implementation: 0 mismatches in 172,463 comparisons over the six books. |
| Built for religious texts | Honorifics («ﷺ», «رضي الله عنه», «عز وجل» only after «الله»), a nameKey for narrator names, and an experimental isnad/matn splitter measured on hand-labelled hadiths. |
| Small and dependency-free | The core JS bundle is 7.9 KB min+gzip with no runtime dependencies; the Dart, PHP and Rust packages have none either. |
Packages
| Language | Package | Install | Docs |
|---|---|---|---|
| Dart / Flutter | nass |
dart pub add nass |
packages/dart |
| TypeScript / JavaScript | @samt-qa/nass |
npm i @samt-qa/nass |
packages/js |
| PHP ≥ 8.3 | samt/nass |
composer require samt/nass |
packages/php |
| Rust ≥ 1.70 | nass |
cargo add nass |
packages/rust |
Note
Version 0.3.0. Published on pub.dev, npm, Packagist and crates.io. All test vectors are reviewed and approved (SPEC §12–14).
Quick start
TypeScript / JavaScript
import { normalize, findAll, highlight, nameKey } from "@samt-qa/nass"; import { splitIsnad } from "@samt-qa/nass/isnad"; // experimental, separate entry normalize("إِنَّمَا الْأَعْمَالُ بِالنِّيَّاتِ"); // "انما الاعمال بالنيات" findAll("إِنَّمَا الْأَعْمَالُ بِالنِّيَّاتِ", "الاعمال"); // [{ start: 9, end: 21, text: "الْأَعْمَالُ" }] highlight(text, "الاعمال", { wholeWord: true }); // wraps matches in <mark> nameKey("أبي عبدِ الله البخاريّ رحمه الله"); // "ابو عبدالله البخاري"
Dart / Flutter
import 'package:nass/nass.dart'; normalize('إِنَّمَا الْأَعْمَالُ بِالنِّيَّاتِ'); // 'انما الاعمال بالنيات' findAll(text, 'الاعمال', matchMode: MatchMode.allWords); highlight(text, 'الاعمال');
PHP
use Samt\Nass\Nass; Nass::normalize('إِنَّمَا الْأَعْمَالُ بِالنِّيَّاتِ'); // 'انما الاعمال بالنيات' Nass::highlight($text, 'الاعمال'); // offsets are code points in PHP
Rust
use nass::*; normalize("إِنَّمَا الْأَعْمَالُ بِالنِّيَّاتِ", &Options::default()); // "انما الاعمال بالنيات" find_all(text, "الاعمال", &SearchOptions::default()); // offsets are UTF-8 bytes in Rust highlight(text, "الاعمال", &HighlightOptions::default());
Normalization levels
| Level | Use | What it does |
|---|---|---|
clean |
storage, display | Removes tatweel and invisible marks, decomposes presentation forms (except honorific ligatures), unifies whitespace, composes Arabic hamza/madda. The text still looks the same. |
search (default) |
search, indexing | Everything in clean, plus: expands «ﷺ» and «﷽», removes diacritics and Quranic marks, unifies alef/yaa/kaf/haa variants, converts Arabic and Persian digits, turns punctuation into spaces, lowercases ASCII. With honorifics: strip it also removes written honorifics. |
loose |
tolerant search | Everything in search, plus: «ة» → «ه», «ؤ» → «و», «ئ» → «ي», removes lone «ء». Merges some different words on purpose. |
Every rule, with its rationale and Unicode ranges, is in spec/SPEC.md.
splitIsnad (experimental)
splitIsnad("حدثنا أبو بكر بن أبي شيبة حدثنا عبد الله بن نمير عن هشام بن عروة عن أبيه عن عائشة أن النبي صلى الله عليه وسلم قال الحمى من فيح جهنم فابردوها بالماء"); // { isnad: "حدثنا … أن النبي صلى الله عليه وسلم قال", // matn: "الحمى من فيح جهنم فابردوها بالماء", confidence: 0.9, // narrators: [{ name: "أبو بكر بن أبي شيبة", form: "حدثنا" }, …, { name: "عائشة", form: "عن" }] } // (سنن ابن ماجه 3462, Open-Hadith-Data)
| Set | Cases | Correct boundary |
|---|---|---|
| Design (dev) | 500 | 98.0% |
| Held out (test, new sample, measured once) | 300 | 88.3% |
The 0.2 algorithm scores 86.7% on the same held-out set. Every held-out result so far is close to 88%, so splitIsnad stays experimental until a held-out set reaches 90%.
Per-book results and the error analysis are in docs/corpus.md.
Important
splitIsnad is a heuristic parser, not a scholar. narrators does not identify people, and two names with the same nameKey are not necessarily the same person.
Quality and testing
| Check | Result |
|---|---|
| Shared test vectors (normalize, offsets, search, names, isnad) | 1,554 cases, all four packages |
| Differential test vs. independent oracle (six books) | 0 mismatches / 172,463 (Dart and Rust) |
| Line coverage | Dart 99.3% · TypeScript 99.6% · PHP 97.9% · Rust 98.0% |
| CI matrix | Dart 3.13 · Node 20 / 22 / 24 · PHP 8.3 / 8.4 / 8.5 · Rust stable / 1.70 on Linux and Windows |
| Property tests | idempotence and clean ⊂ search on random input; Rust also checks search ⊂ loose, offset maps and match spans |
Performance
The first 1,048,576 UTF-16 code units (about 2 MB of UTF-8, 1,658 hadiths) of the fully vocalized Ṣaḥīḥ al-Bukhārī from Open-Hadith-Data, in file order. All rows were measured in one session on one desktop machine (Windows 11), best of two runs:
| Runtime | clean | search | loose |
|---|---|---|---|
Rust 1.95, release (cargo bench) |
11 ms | 15 ms | 15 ms |
| Dart 3.13 VM | 22 ms | 30 ms | 32 ms |
| Node 22 | 36 ms | 36 ms | 39 ms |
PHP 8.5, JIT on (opcache.jit=tracing) |
93 ms | 135 ms | 139 ms |
| PHP 8.5, no JIT | 248 ms | 380 ms | 396 ms |
Rust with the offset map (normalize_with_map): 18 / 21 / 22 ms. To reproduce: node tools/fetch-corpus.mjs && node tools/bench-text.mjs, then each package's benchmark (packages/*/benchmark, packages/js/scripts/bench.mjs, cargo bench in packages/rust). The core JS bundle is 7.9 KB min+gzip; @samt-qa/nass/isnad adds about 2 KB and reuses the core.
Compared with other libraries
From a review in November 2026 (all MIT-licensed). nass was written from scratch; ideas borrowed are credited.
| nass | arabic-normalize (TS) | arabic-normalize (Rust) | arabic_dev_utils (Dart) | |
|---|---|---|---|---|
| Offsets in the original text | ✅ | ✅ | — | — |
| Highlight in vocalized text | ✅ | ✅ | — | — |
| Presentation forms (ﷲ ﻻ ﷺ) | ✅ | ✅ | — | — |
| Quranic marks (dagger alef, pause marks) | ✅ | ✅ | ? | — |
Narrator nameKey, isnad split |
✅ | — | — | — |
| Same behaviour across languages | Dart · TS · PHP · Rust | TS | Rust · Python | Dart |
| Shared test vectors (CC0) | 1,554 | — | — | — |
✅ supported · — not supported · ? not checked in the review
nass's shared vectors were also run against these three libraries; the per-rule results, failures and how to reproduce them are in docs/comparison.md. On the 205 normalize cases each library supports (spec v0.2 vectors), the pass rates were 87% (TS), 37% (Rust) and 32% (Dart); a failure means the output differs from nass's spec, not that the library is wrong for its own purpose.
Repository layout
spec/
SPEC.md the specification (Arabic)
vectors/ shared test vectors (CC0)
tools/
tables.src.json single source of the character tables
*.mjs generators, independent oracle, checks, isnad evaluation
packages/
dart/ js/ php/ rust/ the four implementations
docs/
corpus.md text sources, licences, isnad protocol and results
node tools/gen-tables.mjs --check # tables match the source and Unicode 17 node tools/check-vectors.mjs # structure and properties of every case node tools/fetch-corpus.mjs && node tools/differential.mjs # reference vs. independent oracle node tools/differential.mjs rust # the Rust package vs. the oracle
CI runs all of the above on every push (ci.yml).
Contributing
- Behaviour changes start in
spec/SPEC.mdandspec/vectors/; all four packages then follow. - Never add Quran or hadith text to the tests without a cited source.
- Commit messages are in English; documentation is in both languages.
License
- Code: MIT © 2026 Samt Software.
- Test vectors: CC0. The religious texts quoted in them belong to no one; each case cites its source.
- Hadith data: Open-Hadith-Data (ODbL / DbCL).
☕ ادعم المشروع / Support the project
إذا أفادك هذا المشروع، يمكنك دعم تطويره عبر PayPal. If this project helps you, you can support its development via PayPal.