Categories: None [Edit]

arachnidish

https://rubygems.org/gems/arachnidish
https://github.com/csphere/Arachnid
Arachnidish is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Total

Ranking: 149,377 of 187,603
Downloads: 3,730

Daily

Ranking: 145,949 of 187,591
Downloads: 0

Depended by

RankDownloadsName

Depends on

RankDownloadsName
161,089,000,676nokogiri
190241,404,883typhoeus
3,3093,844,954domainatrix
5,2191,261,839bloomfilter-rb

Owners

#GravatarHandle
1iconcsphere