sentencepiece

https://github.com/yoshoku/sentencepiece.rb

sentencepiece.rb provides Ruby bindings for the SentencePiece, an unsupervised text tokenizer and detokenizer for neural network-based text generation.