Categories: Web Crawling [Edit]

spidr

https://rubygems.org/gems/spidr
https://github.com/postmodern/spidr
Spidr is a versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Total

Ranking: 8,386 of 187,988
Downloads: 429,092

Daily

Ranking: 6,554 of 187,967
Downloads: 81

Depended by

RankDownloadsName
8,487416,521katello
14,498138,365validate-website
25,73654,705esearchy
26,06853,668wayback_archiver
47,14624,022monet
67,37514,730hacker-gems
67,93914,544contentar
76,12112,276ronin-web-spider
120,0645,602spidr_cli
138,9534,280stupid_crawler
141,4624,138site_health
160,1463,217snackhack2

Depends on

RankDownloadsName
13,048,429,741bundler
161,099,518,884nokogiri
135320,171,317base64

Owners

#GravatarHandle
1iconpostmodern