Categories: None [Edit]

arachnidish

https://rubygems.org/gems/arachnidish
https://github.com/csphere/Arachnid
Arachnidish is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Total

Ranking: 149,754 of 186,961
Downloads: 3,634

Daily

Ranking: 143,384 of 186,954
Downloads: 3

Depended by

RankDownloadsName

Depends on

RankDownloadsName
161,065,616,469nokogiri
190235,889,866typhoeus
3,2963,797,536domainatrix
5,1971,247,252bloomfilter-rb

Owners

#GravatarHandle
1iconcsphere