cookler

https://github.com/gastounage/cookler

Cookler is a Ruby library that provides spidering and analysing features. It helps you to quickly write a program to retrieve content and statistics of any website. The API is based on Anemone (http://anemone.rubyforge.org) and use MongoDB as storage solution. Cookler has a multi-threaded and easy-to-use design. The retrieved data and generated stats are stored in 'cookler-db'.