Smarter Rate Limiting Using IP Data
Plain IP rate limiting is easy to bypass. Combining connection type, ASN, and threat score makes it far more effective.
Standard rate limiting counts requests per IP. It's a good baseline, but sophisticated scrapers and abusers bypass it trivially by rotating IPs within the same proxy pool or using residential proxies where millions of IPs are available.
ASN-level rate limits
In addition to per-IP limits, apply per-ASN limits. A scraper rotating through a datacenter's IP pool will still hit the ASN-level cap. This is especially effective against cloud-based scrapers.
Differentiate by connection type
Residential IPs and mobile IPs represent real users. Datacenter IPs almost never do (for consumer products). Apply stricter limits or require authentication for datacenter traffic.
Threat score as a multiplier
Rather than binary allow/block, use the threat score to adjust limits dynamically:
- Score 0–30: full rate limit (e.g., 1000 req/hr)
- Score 31–60: reduced limit (100 req/hr)
- Score 61–80: challenge (CAPTCHA) before allowing
- Score 81+: block or require authentication
Example implementation
= ();
= 1000;
= ->threat_score;
= match(true) {
>= 81 => 0,
>= 61 => 10,
>= 31 => 100,
default => ,
};This pattern catches most abuse while keeping friction low for legitimate users.