Command-Line Interface¶

disarm provides a command-line tool for transliteration, slugification, normalization, and text processing. It reads from arguments or stdin and writes to stdout, making it composable with other Unix tools.

Installation¶

pip install disarm

After installation, the disarm command is available:

disarm t "café"
# cafe

You can also run it as a Python module:

python -m disarm t "café"

Commands¶

Every command has a short alias for faster typing in pipelines.

Command	Alias	Description
transliterate	t	Convert Unicode text to ASCII
slugify	s	Generate URL-safe slugs
normalize	n	Apply Unicode normalization
pipeline	p	Run multi-step text processing
demojize	d	Expand emoji to text descriptions

---

transliterate (t)¶

Convert Unicode text to ASCII using language-aware transliteration tables.

disarm t "café résumé"
# cafe resume

disarm t "Москва"
# Moskva

disarm t "北京市"
# bei jing shi

Options:

--lang CODE

Apply language-specific transliteration rules. Use auto for script-based detection.

disarm t --lang de "Ärger über Ölförderung"
# Aerger ueber Oelfoerderung

disarm t --lang auto "Москва"
# Moskva

--target CODE

Reverse transliteration — convert romanized Latin text back to a native script. Mutually exclusive with --lang.

disarm t --target ru "Moskva"
# Москва

disarm t --target el "Athina"
# Αθηνα

--tones

Include tone marks in Chinese pinyin output.

disarm t --tones "北京"
# běi jīng

--strict-iso9

Use the scholarly ASCII (ISO 9-style) transliteration for Cyrillic. NOTE: ASCII digraphs (zh/ch/sh), not the diacritic ISO 9:1995 standard.

disarm t --strict-iso9 "Юрий"
# Ûrij

--gost7034

Use GOST R 7.0.34 transliteration for Cyrillic.

---

slugify (s)¶

Generate URL-safe slugs from Unicode text.

disarm s "Hello, World!"
# hello-world

disarm s "Ärger im Büro"
# arger-im-buro

disarm s --lang de "Ärger im Büro"
# aerger-im-buero

Options:

--lang CODE

Language-specific transliteration before slugification.

--separator CHAR

Separator character (default: -).

disarm s --separator "_" "Hello World"
# hello_world

--max-length N

Maximum slug length.

disarm s --max-length 10 "A very long blog post title"
# a-very-lon

---

normalize (n)¶

Apply Unicode normalization.

disarm n "café"
# café  (NFC — composed form, the default)

disarm n --form NFKC "fi"
# fi

disarm n --form NFD "é"
# é  (two codepoints: e + combining acute accent)

Options:

--form {NFC,NFD,NFKC,NFKD}

Normalization form (default: NFC).

---

pipeline (p)¶

Run multiple processing steps in a single pass.

disarm p --steps "normalize,fold_case,transliterate" "Héllo WÖRLD"
# hello world

disarm p --steps "normalize,strip_accents,fold_case" "Café Résumé"
# cafe resume

Options:

--steps STEPS

Comma-separated list of processing steps (required).

Available steps: normalize, transliterate, fold_case, collapse_whitespace, strip_accents, confusables, strip_control, strip_zero_width, demojize, strip_bidi, strip_zalgo, strip_pua, strip_plane14, resolve_deletions, resolve_cr, digit_policy, strict_iso9, gost7034.

--form FORM

Normalization form when using the normalize step.

--digit-policy {numeric,tr39,preserve}

The policy the confusables step folds digits under, applied by the digit_policy step entry (default: numeric). Refused, with exit 1, when the steps carry no confusables (#646).

disarm p --steps "confusables" "g੦ogle"                                   # GURMUKHI ZERO read as a digit
# g0ogle
disarm p --steps "confusables,digit_policy" --digit-policy tr39 "g੦ogle"  # ...as the letter it resembles
# google

---

demojize (d)¶

Expand emoji to their text descriptions.

disarm d "Hello 😀 World 🌍"
# Hello grinning face World globe showing Europe-Africa

---

scan (sc)¶

Walk files and directories and report every anomaly inspect_anomalies finds, located by line and column.

disarm scan src/
# src/auth.py:41:17: bidi: "user\u202egpj.exe" contains the bidi override U+202E
# src/i18n.py:3:1: invisible: "ad\u200bmin" contains U+200B ZERO WIDTH SPACE
# scanned 212 file(s), 2 finding(s)

disarm scan . --fail            # exit 1 if anything is found — for CI
disarm scan src/ --json         # machine-readable, with line and column

inspect_anomalies has always returned everything a scanner needs — a kind, a span, evidence and a plain-language reason — and until #704 there was no way to point it at a file.

What the walk does, and does not do:

• Respects git's ignore rules, all three sources. git reads .gitignore in the scanned directory and every parent up to the repository root, .git/info/exclude, and the global core.excludesFile. A scanner that reads only the nearest file gives different answers for disarm scan src/ and disarm scan . on one tree. disarm asks git check-ignore rather than reimplementing the rule, so the two cannot disagree. --no-gitignore turns it off; outside a repository there is nothing to ask and the scan simply has no ignore rules.
• Skips directories that hold no hand-written source — node_modules, __pycache__, .venv, .terraform and the like. build, dist, out, target, bin and vendor are not skipped: they are generated in some projects and hand-written in others, and a scanner that skips them by name reports clean on a tree it never read.
• Never follows symlinks, so a scan stays inside the tree it was pointed at.
• Skips binary and non-UTF-8 files silently. They are not errors; they are not text.

Options:

--json

Emit {"findings": [...], "scanned": N, "unreadable": [...]}. Each finding carries path, line, column, kind, reason and token. line and column are 1-based, and column counts characters — what an editor's gutter shows — not the byte offsets the library reports.

--fail

Exit 1 when anything is found. Without it a scan with findings still exits 0, so the command can be used to look without gating.

--no-gitignore

Scan everything under the paths, ignoring git's rules.

--sarif

Emit SARIF 2.1.0 instead of text, so findings land in GitHub's Security tab and as annotations on the pull request that introduced them. Mutually exclusive with --json. One rule per kind, one result per finding, and the disarm fingerprint in partialFingerprints["disarm/v1"] so a consumer that baselines on its own keys on the same identity this tool does.

--write-baseline FILE

Record every current finding's fingerprint to FILE and exit. This is what lets the check go on at all: a repository with years of history is not clean on its first scan, and requiring it to be clean first is the order that stops adoption.

--baseline FILE

Suppress findings whose fingerprint is in FILE. They are counted (N baselined in the summary) and not shown. An entry that matches no current finding is reported on stderr as stale rather than quietly kept, so the file shrinks as the tree is cleaned; rewrite it with --write-baseline to drop them.

disarm scan --write-baseline .disarm-baseline.json .     # once, today
disarm scan --fail --baseline .disarm-baseline.json .    # in CI, from now on

Fingerprints survive an edit¶

A fingerprint is the file, what was found, and which occurrence of it this is — never the line. The naive identity (file + line + column) breaks on the first commit that inserts a paragraph, because every finding below the insertion looks new. Keyed on the occurrence index, inserting text above a finding does not raise a second alert for it.

The honest limit belongs beside the rule. Two occurrences of the same finding in one file are told apart only by their order, so inserting a second occurrence above a recorded first accepts the new one and reports the old. The count stays right and nothing is silently dropped, but which occurrence is named can swap. Fingerprints are computed on the path as scanned, so a baseline written from the repository root has to be applied from the repository root.

SARIF levels¶

smuggled — a run that decodes to readable text — is the one error: it is the one finding that needs no threshold to interpret. Every other kind is warning, because each reports a technical fact and leaves the judgement to the caller, which is the library's stated stance. The map lives in the writer rather than on the library's AnomalyKind (#705 item 4: a severity on the enum is a public API addition and its own change), and a test asserts it covers every kind the library can produce.

In GitHub Actions¶

- name: Scan for hidden characters
  id: scan                           # the last step reads this step's outcome
  run: pip install disarm && disarm scan --sarif --fail --baseline .disarm-baseline.json . > disarm.sarif
  continue-on-error: true            # the upload below must run even when the scan fails

- name: Upload to the Security tab
  uses: github/codeql-action/upload-sarif@v3
  if: always()                       # ...and this is what makes sure it does
  with:
    sarif_file: disarm.sarif

- name: Fail on new findings
  if: steps.scan.outcome == 'failure'
  run: exit 1

The detail that matters: the upload step has to run before any failing exit, or it never runs at all. A scan that exits 1 and takes the job with it uploads nothing, so the findings that caused the failure are the ones nobody sees. continue-on-error on the scan and if: always() on the upload keep the order right; the last step restores the failure.

Exit codes — something found is not something failed to read, and the codes keep them apart:

Code	Meaning
0	Scanned; nothing found, or found without --fail
1	Findings, with --fail
2	Invalid arguments (argparse)
3	A path could not be read — reported on stderr, scan of the rest still printed. Also a --baseline file that cannot be read or a --write-baseline file that cannot be written

---

Piping and stdin¶

All commands accept input from stdin when no positional argument is given. This makes disarm composable with other tools:

# Process a file
cat names.txt | disarm t

# Chain with other commands
echo "Ünïcödé Tëxt" | disarm t
# Unicode Text

# Slugify each line of a file
while IFS= read -r line; do
    echo "$line" | disarm s
done < titles.txt

# Use with xargs
cat words.txt | xargs -I{} disarm t "{}"

# Combine with sort/uniq for deduplication
cat entries.txt | disarm t | sort -u

Exit codes¶

Code	Meaning
0	Success
1	No input provided (no argument and no stdin)
2	Invalid arguments (unknown command, bad option)
3	scan only: a path could not be read (see scan)