Categories: None [Edit]
tiktoken_ruby
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs.
Total
Ranking: 2,746 of 196,550
Downloads: 7,270,927
Daily
Ranking: 1,622 of 196,430
Downloads: 14,135
Downloads Trends
Ranking Trends
Num of Versions Trends
Popular Versions (Major)
Popular Versions (Major.Minor)
Depended by
| Rank | Downloads | Name |
|---|---|---|
| 9,089 | 434,049 | discourse_ai-tokenizers |
| 14,379 | 162,152 | slack-smart-bot |
| 41,034 | 31,465 | jekyll-chatgpt-translate |
| 73,964 | 13,900 | smart_prompt |
| 81,050 | 12,107 | openai.rb |
| 85,606 | 11,121 | llm_conductor |
| 108,105 | 7,523 | kward |
| 116,402 | 6,518 | dorian-commit |
| 126,785 | 5,449 | htm |
| 181,199 | 2,118 | myrr-rb |
| 182,286 | 1,973 | roseflow-tiktoken |
| 184,497 | 1,601 | nanogpt |
| 187,593 | 1,115 | rubocop-prompt |
| 187,743 | 1,095 | ask-tokens |
| 188,907 | 942 | ghostest |
| 189,221 | 905 | ask-token-usage |
| 189,675 | 854 | ruby_llm-tokenizer |
| 192,601 | 535 | crimson-code |
Depends on
| Rank | Downloads | Name |
|---|---|---|
| 1,315 | 35,856,361 | rb_sys |
Owners
| # | Gravatar | Handle |
|---|---|---|
| 1 |  | iapark |
| 2 | | gjtorikian |
xmisao