Categories: None [Edit]

arachnid

https://rubygems.org/gems/arachnid
https://github.com/dchuk/Arachnid
Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Total

Ranking: 27,420 of 193,540
Downloads: 53,501

Daily

Ranking: 54,393 of 193,523
Downloads: 2

Depended by

RankDownloadsName

Depends on

RankDownloadsName
151,208,435,842nokogiri
187273,784,161typhoeus
3,4004,090,650domainatrix
4,9291,636,897bloomfilter-rb

Owners

#GravatarHandle
1icondchuk