twfilter

https://github.com/taiwan-corpora/twfilter

Provenance analysis of Chinese text in pure Ruby: MOE character-chart tiering, OpenCC-derived simplified round-trip detection, mainland traditional orthography, corpus-verified lexical origin markers graded by attestation ratio, Cantonese particles, erhua windowing, literary Chinese density, and positive Taiwan evidence over lexicon, particles, institutions and the 有 + V perfective. Punctuation normalization to 教育部《重訂標點符號手冊》 and sentence segmentation over terminator-closer classes. Every rejection is reported as a typed finding. No dependencies, no native extensions, deterministic output.