Categories: None [Edit]
tokenizer
A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.
Total
Ranking: 10,006 of 198,774
Downloads: 363,046
Daily
Ranking: 6,436 of 198,739
Downloads: 599
Downloads Trends
Ranking Trends
Num of Versions Trends
Popular Versions (Major)
Popular Versions (Major.Minor)
Depended by
| Rank | Downloads | Name |
|---|---|---|
| 8,104 | 587,819 | metanorma-iso |
| 71,808 | 14,889 | social_tokenizer |
| 82,428 | 12,089 | smalltext |
| 83,215 | 11,917 | jekyll-related-posts |
| 116,884 | 6,640 | meiou |
| 141,119 | 4,648 | shalmaneser-frappe |
| 151,040 | 4,111 | chunkify |
| 153,396 | 3,999 | TacTalk |
| 178,905 | 2,489 | sentimentanalyzer |
Depends on
| Rank | Downloads | Name |
|---|
Owners
| # | Gravatar | Handle |
|---|---|---|
| 1 | arbox |