AI Text Cleaner Methodology — Rules, Defaults, and Limitations

What the local cleaner does

HumanFlow's local cleaner applies deterministic character and pattern transformations. Given the same input, enabled rule set, and dash options, the engine produces the same output and ordered change list. It does not ask a language model to infer meaning. Each change records its rule ID, category, source and output ranges, removed source value, replacement value, Unicode details when relevant, and whether the rule is considered destructive.

Safe defaults target formatting artifacts that usually do not express wording: supported hidden controls, non-breaking spaces, tabs, repeated horizontal spaces, trailing whitespace, and excessive blank lines. Optional rules can alter Markdown or punctuation and therefore start disabled on the general AI text cleaner.

Rule registry

“Safe” does not mean correct for every document. Tabs and repeated spaces may encode code indentation or table alignment. Zero-width joiners and direction marks may be meaningful in multilingual text. Every rule remains independently selectable, and Undo restores the original input held in browser state.

Overlap and change ordering

The engine first collects proposed edits against the original source, orders them by source position and rule priority, and rejects overlapping lower-priority edits. It then builds the output from left to right and records output ranges after each accepted replacement. This prevents two enabled rules from editing the same source span twice and keeps source ranges stable for review.

For example, a run containing non-breaking spaces can match both repeated-space and single-NBSP rules. The higher-priority run replacement is accepted, and overlapping single-character replacements are skipped. The result is one reported transformation for that source span rather than contradictory edits.

Reproducible fixtures

The checked-in test corpus includes empty input, clean prose, mixed Unicode, Markdown, punctuation, multilingual text, code, JSON, and a large deterministic document. Additional fixtures verify exact source/output ranges, named U+200B detection, tab normalization, code-fence preservation of contents, Markdown-link anchor preservation, and every specialized tool preset.

One minimal fixture is A[U+200B]B[U+00A0]C. With safe defaults, the expected output is AB C. A clean sentence containing smart quotes and an em dash remains byte-for-byte unchanged because punctuation rules are off. A Markdown heading also remains unchanged under safe defaults; the focused formatting remover deliberately enables its Markdown preset.

Specialized tool policies

The four focused tools share the same engine rather than rebuilding transformations:

• Remove ChatGPT formatting enables Markdown and spacing rules and includes code-fence and link handling.
• Invisible character remover exposes only hidden Unicode and NBSP rules, with code-point evidence.
• Em dash remover exposes independent em/en dash controls and starts with em-dash-to-hyphen selected.
• Fix AI spacing exposes NBSP, tab, repeated-space, trailing-space, and blank-line rules.

Automated policy tests require four distinct default-rule signatures, substantial unique content, a page-specific sample, and an output change when each preset processes its own sample. This guards against publishing four copy-only variants.

Privacy boundary

Local cleanup runs in the browser and has a source-level test forbidding text-bearing fetches and server actions from its client island. Its analytics event allowlist contains aggregate rule and count data only. See product privacy for the humanizer, checker, account, billing, and optional analytics boundaries.

The AI humanizer is a separate product contract. It can change words and may send submitted text through the server to a configured model provider. The writing-pattern checker is also separate: it runs locally but reports heuristics, not authorship.

Limitations

The cleaner cannot establish whether text was written by a human or a model. It cannot promise to remove statistical watermarking, improve search rankings, bypass AI detectors, or preserve semantics after every optional punctuation or Markdown change. It recognizes only its documented patterns. Nested Markdown, complex link syntax, custom markup, rich-text styling, and application-specific clipboard metadata can fall outside those patterns.

Always validate code, JSON, tables, bidirectional text, and language-specific shaping in the destination application. A visible diff and deterministic output make review possible; they do not replace that review.

Version and change policy

The current local formatting engine schema version is 1. Rule IDs are stable API values used by page presets, analytics allowlists, tests, and governance content. A behavior change must update its fixture expectations and this methodology in the same release. New destructive rules must default to off. Removing or renaming a rule requires an explicit migration because saved policies and centralized SEO observations may refer to its ID.

This methodology was last reviewed against the repository implementation on 2026-09-05. Future review dates must reflect a real code and configuration audit, not an automated timestamp.