Categories: None [Edit]

arachnidish

https://rubygems.org/gems/arachnidish
https://github.com/csphere/Arachnid
Arachnidish is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Total

Ranking: 149,794 of 186,715
Downloads: 3,588

Daily

Ranking: 118,143 of 186,700
Downloads: 2

Depended by

RankDownloadsName

Depends on

RankDownloadsName
181,053,043,305nokogiri
189233,300,180typhoeus
3,2893,775,182domainatrix
5,1781,239,638bloomfilter-rb

Owners

#GravatarHandle
1iconcsphere