Categories: None [Edit]

tokenizer

https://rubygems.org/gems/tokenizer
https://github.com/arbox/tokenizer
A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.

Total

Ranking: 10,383 of 186,954
Downloads: 260,584

Daily

Ranking: 9,997 of 186,947
Downloads: 41

Depended by

RankDownloadsName
8,322420,090metanorma-iso
70,24313,483social_tokenizer
80,88310,916jekyll-related-posts
97,0278,161smalltext
101,4077,582valkey-objects
137,9504,205shalmaneser-frappe
151,1503,575TacTalk
154,0353,466meiou
173,2172,185sentimentanalyzer

Depends on

RankDownloadsName

Owners

#GravatarHandle
1iconarbox