Categories: None [Edit]
tiktoken_ruby
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs.
Total
Ranking: 2,854 of 192,137
Downloads: 6,142,065
Daily
Ranking: 2,092 of 192,103
Downloads: 5,420
Downloads Trends
Ranking Trends
Num of Versions Trends
Popular Versions (Major)
Popular Versions (Major.Minor)
Depended by
| Rank | Downloads | Name |
|---|---|---|
| 10,712 | 271,329 | discourse_ai-tokenizers |
| 14,120 | 155,161 | slack-smart-bot |
| 44,887 | 26,807 | jekyll-chatgpt-translate |
| 106,013 | 7,476 | openai.rb |
| 125,396 | 5,312 | dorian-commit |
| 133,891 | 4,741 | llm_conductor |
| 146,187 | 4,055 | htm |
| 180,490 | 1,685 | roseflow-tiktoken |
| 183,720 | 1,118 | nanogpt |
| 185,326 | 895 | rubocop-prompt |
| 185,770 | 836 | ghostest |
Depends on
| Rank | Downloads | Name |
|---|---|---|
| 1,342 | 32,552,999 | rb_sys |
Owners
| # | Gravatar | Handle |
|---|---|---|
| 1 |  | iapark |
| 2 | | gjtorikian |
xmisao