Categories: None [Edit]

tokenizer

https://rubygems.org/gems/tokenizer
https://github.com/arbox/tokenizer
A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.

Total

Ranking: 10,006 of 198,774
Downloads: 363,046

Daily

Ranking: 6,436 of 198,739
Downloads: 599

Depended by

RankDownloadsName
8,104587,819metanorma-iso
71,80814,889social_tokenizer
82,42812,089smalltext
83,21511,917jekyll-related-posts
116,8846,640meiou
141,1194,648shalmaneser-frappe
151,0404,111chunkify
153,3963,999TacTalk
178,9052,489sentimentanalyzer

Depends on

RankDownloadsName

Owners

#GravatarHandle
1iconarbox