Categories: None [Edit]

arachnidish

https://rubygems.org/gems/arachnidish
https://github.com/csphere/Arachnid
Arachnidish is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Total

Ranking: 150,483 of 193,540
Downloads: 3,916

Daily

Ranking: 103,856 of 193,523
Downloads: 0

Depended by

RankDownloadsName

Depends on

RankDownloadsName
151,208,435,842nokogiri
187273,784,161typhoeus
3,4004,090,650domainatrix
4,9291,636,897bloomfilter-rb

Owners

#GravatarHandle
1iconcsphere