Categories: None [Edit]

robots-audit

https://rubygems.org/gems/robots-audit
robots-audit fetches a site's robots.txt (following redirects), parses it per the robots exclusion protocol, and reports actionable findings: parse violations, orphan Disallow lines, missing wildcard User-agent coverage, missing sitemap declarations, unexpected HTTP statuses, and oversize files. Zero runtime dependencies — stdlib only (net/http, uri, optparse, json). Use it for SEO hygiene checks, crawl-budget reviews, and CI gates.

Total

Ranking: 194,579 of 198,817
Downloads: 604

Daily

Ranking: 73,023 of 198,797
Downloads: 3

Depended by

RankDownloadsName

Depends on

RankDownloadsName

Owners

#GravatarHandle
1icondeveloapps-marky