Categories: None [Edit]

arachnidish

https://rubygems.org/gems/arachnidish
https://github.com/csphere/Arachnid
Arachnidish is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Total

Ranking: 149,768 of 187,267
Downloads: 3,678

Daily

Ranking: 138,814 of 187,255
Downloads: 0

Depended by

RankDownloadsName

Depends on

RankDownloadsName
151,077,724,493nokogiri
189238,638,544typhoeus
3,3073,821,613domainatrix
5,2051,255,519bloomfilter-rb

Owners

#GravatarHandle
1iconcsphere