Categories: None [Edit]

cobweb

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Total

Ranking: - of 0

Downloads: -

Daily

Ranking: - of 0

Downloads: -

Downloads Trends

Ranking Trends

Num of Versions Trends

Popular Versions (Major)

Popular Versions (Major.Minor)

Depended by

Rank	Downloads	Name
33,923	39,897	cohesion
62,191	17,513	applied_css
114,616	2,297	rawl
163,222	3,265	seo_site_check

Depends on

Rank	Downloads	Name
8	1,354,615,159	rake
10	1,310,757,635	json
15	1,219,645,394	nokogiri
16	1,212,083,202	addressable
20	1,164,651,127	rspec-core
29	993,218,487	rspec
83	565,294,940	redis
143	347,158,118	sinatra
158	323,138,951	sidekiq
248	206,852,430	thin
249	206,608,771	redis-namespace
263	200,247,772	haml
330	155,741,218	slop
971	52,240,931	mock_redis
1,912	16,839,924	coveralls
2,166	12,040,045	bundle-audit

Owners

#	Gravatar	Handle
1		stewartmckee