Categories: None [Edit]
tiktoken_ruby
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs.
Total
Ranking: 2,812 of 193,756
Downloads: 6,565,875
Daily
Ranking: 1,723 of 193,649
Downloads: 11,502
Downloads Trends
Ranking Trends
Num of Versions Trends
Popular Versions (Major)
Popular Versions (Major.Minor)
Depended by
| Rank | Downloads | Name |
|---|---|---|
| 9,721 | 355,547 | discourse_ai-tokenizers |
| 14,297 | 158,261 | slack-smart-bot |
| 42,886 | 29,006 | jekyll-chatgpt-translate |
| 88,356 | 10,277 | openai.rb |
| 106,865 | 7,498 | llm_conductor |
| 121,306 | 5,839 | dorian-commit |
| 133,521 | 4,844 | htm |
| 181,018 | 1,778 | roseflow-tiktoken |
| 183,319 | 1,354 | nanogpt |
| 184,813 | 1,116 | kward |
| 185,718 | 984 | rubocop-prompt |
| 186,543 | 877 | ghostest |
| 189,380 | 564 | ruby_llm-tokenizer |
| 192,289 | 230 | crimson-code |
Depends on
| Rank | Downloads | Name |
|---|---|---|
| 1,336 | 33,875,951 | rb_sys |
Owners
| # | Gravatar | Handle |
|---|---|---|
| 1 |  | iapark |
| 2 | | gjtorikian |
xmisao