rb_lib_text

https://github.com/peoplepattern/rb-lib-text

The tokenization has been tuned to work well with text conventions commonly used in social media such as Twitter, and supports URLs, hashtags, emails and @-mentions cleanly.