Categories: None [Edit]

cobweb

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Total

Ranking: 9,710 of 193,540

Downloads: 355,666

Daily

Ranking: 24,343 of 193,523

Downloads: 10

Downloads Trends

Ranking Trends

Num of Versions Trends

Popular Versions (Major)

Popular Versions (Major.Minor)

Depended by

Rank	Downloads	Name
33,818	39,870	cohesion
62,028	17,499	applied_css
163,007	3,260	seo_site_check

Depends on

Rank	Downloads	Name
8	1,341,038,984	rake
10	1,295,536,967	json
15	1,208,435,842	nokogiri
16	1,199,982,235	addressable
20	1,157,054,801	rspec-core
29	987,418,354	rspec
83	561,296,784	redis
142	344,958,300	sinatra
158	320,156,553	sidekiq
248	206,397,482	thin
249	205,357,465	redis-namespace
259	198,858,857	haml
326	155,244,692	slop
974	51,603,732	mock_redis
1,910	16,730,237	coveralls
2,177	11,797,468	bundle-audit

Owners

#	Gravatar	Handle
1		stewartmckee