Categories: Web Crawling [Edit]

spidr

https://rubygems.org/gems/spidr
https://github.com/postmodern/spidr
Spidr is a versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Total

Ranking: 8,376 of 187,255
Downloads: 418,445

Daily

Ranking: 7,512 of 187,239
Downloads: 70

Depended by

RankDownloadsName
8,504406,533katello
14,484136,089validate-website
25,54554,506esearchy
26,00653,099wayback_archiver
47,27423,626monet
67,75914,396contentar
67,88514,354hacker-gems
78,62611,540ronin-web-spider
119,9105,514spidr_cli
138,8954,210stupid_crawler
141,1624,080site_health
169,4852,480snackhack2

Depends on

RankDownloadsName
12,973,645,220bundler
151,077,310,049nokogiri
149291,050,770base64

Owners

#GravatarHandle
1iconpostmodern