Classes¶

Stateful objects and builders for repeated or specialized text processing.

Text¶

>>> from disarm import Text
>>> Text("Straße").fold_case().value
'strasse'
>>> Text("  hello   world  ").collapse_whitespace().value
'hello world'
>>> str(Text("café").strip_accents())
'cafe'

value: str

normalize(*, form: NormalizationForm = 'NFC') -> Text

normalize_confusables(*, target_script: str = 'latin', digit_policy: str = 'numeric') -> Text

strip_accents() -> Text

transliterate(*, lang: str | None = None, target: str | None = None, errors: ErrorMode = 'replace', replace_with: str = '[?]', strict_iso9: bool = False, gost7034: bool = False, tones: bool = False, context: bool = False) -> Text

fold_case() -> Text

collapse_whitespace() -> Text

strip_control_chars() -> Text

strip_zero_width_chars() -> Text

slugify(*, separator: str = '-', lowercase: bool = True, max_length: int = 0, word_boundary: bool = False, save_order: bool = False, stopwords: Iterable[str] = (), regex_pattern: str | None = None, replacements: Iterable[tuple[str, str]] = (), allow_unicode: bool = False, lang: str | None = None, entities: bool = True, decimal: bool = True, hexadecimal: bool = True, default: str | None = None) -> Text

sanitize_filename(*, separator: str = '_', max_length: int = 255, platform: Platform = 'universal', lang: str | None = None, preserve_extension: bool = True) -> Text

demojize(*, strip_modifiers: bool = False, errors: ErrorMode = 'replace', replace_with: str = '[?]', provider: EmojiProvider | None = None) -> Text

strip_bidi() -> Text

canonicalize() -> Text

security_clean() -> Text

ml_normalize(*, lang: str | None = None, emoji: str = 'cldr', fold_case: bool = True) -> Text

strip_format() -> Text

display_clean() -> Text

canonicalize_strict() -> Text

normalize_user_input() -> Text

strip_obfuscation() -> Text

is_ascii() -> bool

is_case_fold_stable() -> bool

is_canonical(*, preset: str = 'canonicalize') -> bool

is_normalized(*, form: NormalizationForm = 'NFC') -> bool

is_confusable(*, target_script: str = 'latin') -> bool

is_mixed_script() -> bool

has_bidi_conflict() -> bool

has_bidi_control() -> bool

detect_scripts() -> list[Script]

grapheme_len() -> int

grapheme_split() -> list[str]

grapheme_truncate(max_graphemes: int) -> Text

catalog_key(*, lang: str | None = None, strict_iso9: bool = False) -> Text

Usage¶

from disarm import Text

result = (
    Text("Ünïcödé Café ☕")
    .normalize(form="NFKC")
    .demojize()
    .transliterate()
    .strip_accents()
    .fold_case()
    .value
)
assert result == "unicode cafe hot beverage"

Each transform method returns a new Text instance (immutable semantics, matching Python str). Predicates return their native type (bool, list) and do not chain.

Chainable transforms¶

All core transforms are available as methods:

Non-chaining predicates¶

Result extraction¶

Use .value or str() to extract the underlying string:

from disarm import Text

text = Text("café").strip_accents()
assert text.value == "cafe"
assert str(text) == "cafe"

Text supports ==, hash(), len(), and bool() — comparing against the underlying string value.

Slugifier¶

>>> s = Slugifier(separator="_", lang="de")
>>> s("Ärger im Büro")
'aerger_im_buero'

Usage¶

from disarm import Slugifier

slug = Slugifier(separator="_", lang="de", max_length=50)
assert slug("Ärger im Büro") == "aerger_im_buero"
assert slug("Über den Wolken") == "ueber_den_wolken"

# Auto-detect language from script
auto_slug = Slugifier(lang="auto")
assert auto_slug("Москва") == "moskva"

Accepts all the same parameters as slugify(). Construct once, call many times.

UniqueSlugifier¶

>>> u = UniqueSlugifier()
>>> u("My Post")
'my-post'
>>> u("My Post")
'my-post-1'

reset() -> None

Usage¶

from disarm import UniqueSlugifier

unique = UniqueSlugifier()
assert unique("My Post") == "my-post"
assert unique("My Post") == "my-post-1"
assert unique("My Post") == "my-post-2"

unique.reset()  # clear seen slugs
assert unique("My Post") == "my-post"

External uniqueness check¶

def exists_in_db(slug: str) -> bool:
    return db.slugs.filter(slug=slug).exists()


unique = UniqueSlugifier(check=exists_in_db)

The check callback is called for each candidate slug. If it returns True, the slugifier increments the suffix and tries again.

TextPipeline¶

>>> pipe = TextPipeline(normalize="NFC", fold_case=True, collapse_whitespace=True)
>>> pipe("  Héllo  WÖRLD  ")
'héllo wörld'
>>> TextPipeline(confusables=True, digit_policy="tr39")("g੦ogle")
'google'

purpose: str | None

>>> get_pipeline("rag_ingest").purpose
'Normalizing retrieved documents for a RAG index, romanizing legitimate non-Latin text rather than folding homoglyphs onto Latin.'
>>> TextPipeline(fold_case=True).purpose is None
True

>>> {p: get_pipeline(p).purpose for p in list_profiles()}
{'code_context': ...}

steps: list[tuple[str, str | None]]

>>> pipe = TextPipeline(normalize="NFC", fold_case=True)
>>> pipe.steps
[('normalize', 'NFC'), ('fold_case', None)]

explain() -> str

>>> pipe = TextPipeline(normalize="NFC", fold_case=True)
>>> print(pipe.explain())
TextPipeline with 2 steps:
  1. normalize (NFC)
  2. fold_case

Usage¶

from disarm import TextPipeline

pipe = TextPipeline(
    normalize="NFC",
    confusables=True,
    strip_accents=True,
    fold_case=True,
    collapse_whitespace=True,
)

assert pipe("  Héllo Wörld  ") == "hello world"

strip_zalgo is a cap, and 0 is not "off"¶

Every other step here is switched with a boolean. strip_zalgo takes the maximum number of combining marks to allow per base character, so 0 allows none and removes every diacritic in the text — including from words that were never obfuscated. Off is None, which leaves the step out of the compiled pipeline altogether (#958):

from disarm import TextPipeline

text = "Čeština, naïve café"

# None — the default. The step is not compiled in at all.
off = TextPipeline(strip_zalgo=None)
assert off.steps == []
assert off(text) == text

# 0 — a cap of zero marks. Every diacritic goes.
assert TextPipeline(strip_zalgo=0)(text) == "Cestina, naive cafe"

# A small positive cap runs the step and still leaves ordinary accents alone.
assert TextPipeline(strip_zalgo=3)(text) == text

The same literal reads the other way in PRESETS, where ("strip_zalgo", None) names a step that does run, at its default cap.

Execution order¶

Operations execute in this fixed order regardless of construction order:

1. Normalize → 2. Confusables → 3. Demojize → 4. Strip accents → 5. Transliterate → 6. Fold case → 7. Collapse whitespace

Performance¶

The pipeline is pre-compiled at construction. Enabled steps are stored as a bitflag set — only enabled steps execute at call time.

Compatibility aliases (awesome-slugify)¶

These classes provide drop-in replacements for awesome-slugify's Slugify and UniqueSlugify. They accept awesome-slugify's parameter names and map them to native disarm parameters.

See the migration guide for full details.

Slugify¶

from disarm import Slugify
custom = Slugify(to_lower=True)
custom("Hello World")  # => "hello-world"

from disarm import Slugify

# Same API as awesome-slugify
custom = Slugify(to_lower=True)
assert custom("Hello World") == "hello-world"

# Attribute-style configuration (awesome-slugify pattern)
s = Slugify()
s.to_lower = True
s.stop_words = ("the", "a")
s.max_length = 200
assert s("The Big Fox") == "big-fox"

Accepts both awesome-slugify parameter names (to_lower, stop_words, safe_chars, capitalize, pretranslate) and native disarm names (lowercase, stopwords, replacements).

Defaults to to_lower=False (matching awesome-slugify). For python-slugify compatibility (which defaults to lowercase=True), use the native Slugifier class or the slugify() function.

UniqueSlugify¶

from disarm import UniqueSlugify
unique = UniqueSlugify()
unique("My Post")   # => "My-Post"
unique("My Post")   # => "My-Post-1"

reset() -> None

from disarm import UniqueSlugify

unique = UniqueSlugify(to_lower=True)
assert unique("My Post") == "my-post"
assert unique("My Post") == "my-post-1"

unique.reset()
assert unique("My Post") == "my-post"

Extends Slugify with uniqueness tracking. Accepts uids and unique_check parameters from awesome-slugify.

Preconfigured instances¶

Drop-in replacements for awesome-slugify's preconfigured slugifiers:

from disarm import (
    slugify_url,  # lowercase, strips articles, max 200 chars
    slugify_filename,  # underscore separator, preserves -., max 255 chars
    slugify_unicode,  # keeps non-ASCII letters
    slugify_ru,  # Russian transliteration
    slugify_de,  # German transliteration (ä→ae, ö→oe, ü→ue)
    slugify_el,  # Greek transliteration
)

assert slugify_url("The Big Fox") == "big-fox"
assert slugify_de("Ärger im Büro") == "Aerger-im-Buero"
assert slugify_filename("My Report.pdf") == "My_Report.pdf"