Categories: None [Edit]

tokenizer

https://rubygems.org/gems/tokenizer
https://github.com/arbox/tokenizer
A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.

Total

Ranking: 9,886 of 188,756
Downloads: 221,780

Daily

Ranking: 14,996 of 188,734
Downloads: 29

Depended by

RankDownloadsName
7,926333,841metanorma-iso
68,61512,291social_tokenizer
77,64210,249jekyll-related-posts
135,7333,807smalltext
136,1493,786shalmaneser-frappe
150,8683,181TacTalk
170,1771,892sentimentanalyzer
176,234801meiou

Depends on

RankDownloadsName

Owners

#GravatarHandle
1iconarbox