Skip to content

Syllable Counting: 86 Languages, One Algorithm

4 min read Notes

Syllable Counting: 86 Languages, One Algorithm

Every readability formula divides by syllables. Get the count wrong by fifteen percent and every grade you compute is wrong by roughly the same margin, quietly, in a report nobody questions.

Code: ReadSight, MIT licensed.

English codebases handle this with a heuristic: count vowel groups, subtract a silent e, accept the error. It survives a demo and falls apart on a product page where half the words are names.

The alternative is older than most of the web. TeX needed to know where to break words in 1983, and Frank M. Liang solved it with pattern files per language. The same patterns give a syllable count that holds up on real text.

Why the vowel heuristic fails

“Queue” has one syllable and four vowel groups. “Fire” has one or two depending on the accent. “Created” has three, “wanted” has two, and both follow the same spelling shape.

Proper nouns are worse. A page about a product called Aviva gets an extra syllable in every heading, and a page full of API names reads as harder prose than it is.

None of these errors look like errors. A grade of 12.4 instead of 11.1 passes review because nobody has a second opinion on syllable counts.

How the TeX patterns work

Liang’s method stores patterns over letter sequences, and each pattern carries weights between the letters. For German, a pattern covers a handful of characters, and the digits in the pattern say where a break is allowed and where it is forbidden.

The matcher walks the word, applies every pattern that matches at every position, and keeps the highest weight for each gap. Odd weights mean break, even weights mean do not break. Words up to a length limit are special-cased in a small exception list.

For hyphenation this produces typographically correct break points. For syllable counting it produces a segmentation that matches how the language actually sounds, which is what a readability formula wants.

A note on the difference: hyphenation points and syllable boundaries are not identical. “Hyphenation” splits into hy-phen-ation for typesetting and reads as four syllables in most counts. The library exposes both, splitWord() for the TeX points and splitSyllables() for the heuristic count, so a caller can pick the one the formula expects.

The language file is the hard part

The algorithm is twenty lines. The data is the project.

Each of the 86 languages needs its own pattern file, and the quality of that file decides the quality of every score computed from it. A pattern set built for a closely related language produces plausible output with wrong boundaries, which is the worst kind of bug because it does not crash.

That is also why a library is a better idea than a folder of scripts. The per-language data is versioned, tested and shipped together, and a caller asks for de-DE instead of assembling a pattern path.

Normalisation before counting

The counting step is only as good as the text it receives. Four normalisations cover most real inputs:

  • Case folding, since patterns are stored in one case.
  • Apostrophes and hyphens, which either join a word or split it depending on the language.
  • Numbers and units, which have no syllables and should be read aloud or skipped deliberately.
  • Non-breaking spaces and soft hyphens copied from a document.

Skipping this step produces counts that look random. A soft hyphen inside “cooperation” becomes a word boundary, and the sentence gains a word.

What to do with the number

Once syllables are trustworthy, the formulas become comparable. Flesch Reading Ease, Flesch-Kincaid, Gunning Fog, SMOG, Coleman-Liau, ARI and LIX all divide words and syllables in different combinations, and every one of them inherits your counting error.

Two practical rules follow. Compare a page only with pages in the same language and the same formula. And check the sentence count as well as the syllable count, since a sentence splitter that mishandles abbreviations inflates the average words per sentence and moves the grade more than any syllable error.

The readability metrics guide covers the API side: what the response contains, which fields go null when a formula does not support a language, and how to read the statistics before the grade.

If the text arrives from a form and you are not sure of the language, the text language detection guide explains the sample size needed before the guess is reliable.

Background: Frank M. Liang, Word Hy-phen-a-tion by Com-put-er, and the hyphenation patterns maintained for TeX.

Custom work

Need an API that does not exist yet?

Describe the task and the response format. We will estimate the endpoint, the integration and the maintenance.

Start a conversation