Unicode Inflection

An Elixir-native engine for grammatical inflection, based on the data published by the Unicode inflection project. It inflects words and phrases by number, gender, case and definiteness, queries the grammatical properties of a phrase, selects grammatically agreeing articles, selects grammatically correct pronouns, and quantifies nouns so they agree with a count (Russian numeral government, Arabic duals, CJK classifiers) — the building blocks for grammatically correct messages, including MessageFormat 2 :inflect-style functions and unit formatting in Localize.

The engine is a pure-Elixir port of the upstream C++ library. All 31 upstream data-driven inflection conformance suites pass 100% (6,060 cases across English, Spanish, French, German, Portuguese, Italian, Dutch, Danish, Swedish, Norwegian Bokmål, Russian, Ukrainian, Polish, Czech, Serbian, Hebrew, Arabic, Turkish, Finnish, Romanian, Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Punjabi, Tamil, Telugu, Urdu and Korean), and all 25 upstream pronoun suites pass 100% (1,317 cases). A further 17 languages (Bulgarian, Catalan, Greek, Hungarian, Icelandic, Kazakh, Croatian, Lithuanian, Slovak, Oriya, Thai, Vietnamese, Malay, Indonesian, Japanese, Cantonese and Mandarin) ship dictionary and feature data without a language-specific synthesizer.

Installation

Add unicode_inflection to your list of dependencies in mix.exs:

def deps do
  [
    {:unicode_inflection, "~> 0.1"}
  ]
end

Then install the inflection data bundle — a build-time prerequisite, run once per build environment:

mix unicode_inflection.install

The data (about 41 MB of compiled locale artifacts and pronoun tables) is distributed as a single bundle on the Localize CDN, not inside the hex package. The bundle is versioned by the upstream unicode-org/inflection commit it was built from — independently of the library version — so library upgrades only re-download data when the data pin has actually moved. The download is verified against a SHA-256 digest shipped in the package before anything is written. In a Docker build, add RUN mix unicode_inflection.install after mix deps.compile. The install directory is configurable (see Unicode.Inflection.DataDir); by default the data lives in the dependency's own priv/data.

Documentation

• What is inflection? — an introduction to grammatical inflection: features, grammemes, agreement, and why software messages need them.

• User guide — installing, inflecting, querying features, pronouns, and the Concept APIs.

• Grammatical features — every feature and grammeme explained in plain language: what a dative is, what the Finnish inessive does, what :construct means in Arabic.

Usage

The high-level API inflects, queries and selects pronouns in one call:

iex> Unicode.Inflection.inflect("light on the patio", :en, number: :plural)
{:ok, "lights on the patio"}

iex> Unicode.Inflection.inflect("the cat", :en, definiteness: :indefinite)
{:ok, "a cat"}

iex> Unicode.Inflection.inflect("universidad", :es, number: :plural, definiteness: :definite)
{:ok, "las universidades"}

iex> Unicode.Inflection.inflect("новый дом", :ru, case: :instrumental)
{:ok, "новым домом"}

iex> Unicode.Inflection.feature("luces", :es, :number)
{:ok, :plural}

iex> Unicode.Inflection.pronoun(:en, person: :first, case: :genitive)
{:ok, "mine"}

iex> Unicode.Inflection.pronoun(:en, "they", gender: :feminine)
{:ok, "she"}

iex> Unicode.Inflection.quantify("2", "час", :ru, plural: :few)
{:ok, "2 часа"}

iex> Unicode.Inflection.quantify("2", "رسالة", :ar, plural: :two)
{:ok, "رسالتان"}

The lower-level APIs mirror the upstream concepts. Unicode.Inflection.Concept wraps an inflectable string: build a concept, add constraints, query features, render. Unicode.Inflection.PronounConcept selects pronouns from the locale pronoun table and supports reinflecting an existing pronoun while preserving its unconstrained properties, custom pronoun entries, and sound-dependent selection against a referenced concept (French je / j’ elision).

iex> {:ok, concept} = Unicode.Inflection.Concept.new(:de, "Haus", constraints: %{case: :dative, number: :plural})
iex> Unicode.Inflection.Concept.to_speakable_string(concept)
"Häusern"

iex> {:ok, pronoun} = Unicode.Inflection.PronounConcept.new(:en, initial_pronoun: "him")
iex> {:ok, pronoun} = Unicode.Inflection.PronounConcept.put_constraint(pronoun, :case, :nominative)
iex> Unicode.Inflection.PronounConcept.to_speakable_string(pronoun)
"he"

Rendered values are speakable strings: a plain binary when the spoken form equals the printed form, or a {print, speak} tuple when they differ (for example Danish den/dén). An explicit "speak" constraint overrides the spoken channel.

Data

Linguistic data (lexicons, inflection paradigm tables, grammatical feature definitions, pronoun tables) comes from the unicode-org/inflection repository at a pinned commit, compiled into compact per-locale artifacts. Library users install the pre-built bundle with mix unicode_inflection.install; building from upstream sources is a maintainer workflow:

mix unicode_inflection.download   # fetch upstream source data (git-lfs via the GitHub media endpoint)
mix unicode_inflection.generate   # compile priv/data/<locale>.etf artifacts and pronoun tables
mix unicode_inflection.bundle     # package the bundle for CDN upload and record its SHA-256

Artifacts are loaded lazily per locale at runtime: metadata into :persistent_term, the lexicon into an ETS table. Grammeme sets are represented as integer bitmasks without the upstream 64-bit limit. There are no runtime downloads: after mix unicode_inflection.install the library never touches the network.

Performance

Measured with Benchee on Apple silicon (mix run bench/inflection.exs): single-word feature queries and pronoun selection run in about 2 µs; phrase inflection runs in about 8 µs for English and Finnish and about 21 µs for a two-word Russian phrase with case agreement.

Conformance

Every ported language is gated on the upstream data-driven conformance suites (dialog/inflection/XX.xml and dialog/pronoun/XX.xml in the upstream repository). The suites run as part of mix test; deliberate deviations from upstream (all of them data-availability artifacts, not behavior choices) are documented in the repository ROADMAP.

License

The code is licensed under the Apache License, Version 2.0. The linguistic data is derived from the unicode-org/inflection project, licensed under the Unicode License V3.