What Makes Arabic Hard for Language Models

The tokeniser is where the money goes

A model does not see letters or words. It sees tokens: pieces of text produced by a compression algorithm, usually byte-pair encoding, fit to a training corpus. If that corpus was mostly English, sequences like "tion" and " the" earned a slot in the vocabulary. Arabic competed for the leftovers.

The result is fragmentation. An English word of moderate length is often one token; the same meaning in Arabic often arrives in three or four pieces, sometimes split at points that correspond to nothing linguistic, inside the root or between a word and the prefix glued to it. How much worse Arabic comes off depends heavily on which tokeniser you are using.

Three costs follow. Money: billing is per token, so the same conversation in Arabic costs more. Context: a fixed window holds fewer Arabic words. And quality, more tentatively: a word delivered as five arbitrary fragments has to be reassembled from positions carrying no morphological meaning, and work comparing tokenisers has found that downstream performance tracks how well the segmentation fits the language. How much of the Arabic-English gap that accounts for is not settled.

The fix is upstream: fit the tokeniser on enough Arabic that Arabic morphemes earn their own entries. Retrofitting is possible. You can add Arabic tokens to an existing vocabulary, initialise the new embeddings from the old sub-tokens, and continue pretraining, which is what most Arabic adaptations of English models do. It works, but it is a second training run, and the earlier the decision is made properly the cheaper it is.

Root and pattern: one word, an enormous surface

Arabic builds words by threading a consonantal root, usually three letters, through a template. From k-t-b come the verb for writing, the noun for a book, the word for a library, and the writer. The root carries the semantic field, the pattern the grammatical role. Elegant, and also why Arabic has far more distinct surface forms than English for the same content.

Clitics attach on top of the pattern: the definite article, conjunctions, prepositions, the future marker, object and possessive pronouns. One written Arabic word covers what English needs four for, as in "and in their libraries". Verbs inflect for person, number, gender, tense, mood and voice, and the dual is a real grammatical number beside singular and plural.

The evidence for any one form is therefore thin. Subword tokenisation is meant to solve exactly this, and does, but only when the segmentation lands on real morpheme boundaries. When it does not, the model generalises over the wrong pieces.

The vowels that are not written

Arabic short vowels are diacritics, and outside the Quran, children's books and poetry nobody writes them. The same undiacriticised string can be an active verb, a passive verb, or a noun: the bare letters k-t-b can be read as "he wrote", "it was written", or the plural "books", and only the surrounding sentence decides.

Some of this is the ordinary ambiguity every language has; English "lead" and "read" behave similarly. The difference is density. In Arabic it is systematic rather than occasional, because a whole vowel layer is omitted and the model must infer it from neighbouring words. Two cases are worth watching, because context helps least there: passive readings mistaken for active ones, and proper nouns spelled identically to an ordinary word.

It matters after the model too. Most Arabic text-to-speech pipelines put a diacritisation step in front, and diacritisation is itself an open research task rather than a solved preprocessing detail.

Which Arabic, exactly?

Modern Standard Arabic is the language of news, law and most Arabic text on the public web, and it is nobody's mother tongue. What people actually speak, and increasingly write in messages and transcribed voice notes, is a dialect, and the distance is wide enough that a Moroccan and an Iraqi, each using their own, will not fully understand one another.

The gap is not accent: it is vocabulary, morphology, negation, question formation, and enough syntax to matter. Iraqi Arabic marks the progressive with a da- prefix on the verb, has its own negation particles, and carries a thick layer of Turkish and Persian borrowing. Egyptian has the longest history of dialect resources in Arabic NLP, the earliest and largest labelled corpora being Egyptian, while Gulf and Iraqi data has been built later and thinner. Models inherit that history: ask one for Iraqi and it drifts toward Egyptian.

Writing practice compounds it. Dialect has no standardised orthography, so one word appears in several spellings, and Arabizi, Arabic written in Latin letters with digits for the sounds Latin lacks, is effectively a second script to handle. Code-switching with English inside a sentence is normal, not exceptional.

Direction, mixing, and the parts that break outside the model

Arabic runs right to left, but numbers run left to right, and so do the Latin technical terms that fill Arabic writing about software. The Unicode bidirectional algorithm resolves this at display time and mostly works, until it does not. Punctuation next to a direction boundary is the classic failure: a bracket or full stop between an Arabic phrase and a Latin one can land on the wrong side.

This is a rendering problem rather than a model one, and it surfaces wherever model output lands: chat bubbles, terminals, PDF exports, logs. Perfectly correct Arabic can display as garbage because the interface around it was never tested. Isolating Latin and numeric runs helps, as does checking the rendered output rather than the string.

Character normalisation is the quieter version, and it is a choice with no correct answer. Alef with and without hamza, taa marbuta against haa, final yaa against alef maqsura are written inconsistently by real users, so any normaliser collapses distinctions that are sometimes real: it helps retrieval and hurts anything that needs the text back verbatim.

What "Arabic-first" has to mean to be worth saying

Arabic has the speaker count of a major world language and nothing like the matching web presence. Audits of the large multilingual crawl corpora have repeatedly found the Arabic portions, the dialect portions worst of all, carrying machine translation, duplication and mislabelled text, so the effective corpus is smaller than the raw counts suggest. How much smaller nobody has measured cleanly. Cleaning it is expensive, and the cleaning tools were themselves built mostly on English.

"Arabic-first" is now a marketing phrase, and usually it means a system prompt in Arabic wrapped around a model trained like everything else. That helps; prompting genuinely improves register and dialect. But it is a thin layer over an English-shaped foundation.

The claim becomes meaningful when it names something checkable. Does the tokeniser give Arabic a tokens-per-word ratio close to English, and can you show the number? Was the evaluation written in Arabic, or translated from an English benchmark that quietly measures translation quality? Is dialect a target rather than an error, with the dialects named? Are RTL and bidi correctness requirements with tests behind them? Each answer has a price: an Arabic-heavy vocabulary takes room from other languages, native evaluation must be written rather than translated, and dialect work does not transfer between dialects.

We publish benchmark figures for Firas AI from our own evaluation runs, not independent ones, and we say so on the page, which by the standard above is a disclosure and not a defence. What has genuinely improved in the field is narrow and nameable: tokeniser vocabularies in widely used models have grown from around 32,000 entries to well over a hundred thousand, and Arabic words now survive segmentation in fewer pieces. The distance to English is shorter. It has not closed.