How to Build a Simple Proxy Rotator in Python 2026
Build a working proxy rotator in Python from scratch — from a three-line version to a resilient class that retries and retires dead proxies automatically.
If you've ever watched a scraping script sail through ten requests and then slam into a wall of 403 errors, you've met the problem a proxy rotator solves. Sending every request from a single IP is the fastest way to get blocked — and the fix is to spread that traffic across many addresses automatically.
The need is real and growing. With an estimated 47% of internet traffic now automated and major sites rate-limiting a single IP after as few as 10–20 rapid requests, rotation isn't a nice-to-have — it's the difference between scraping ten pages and ten million.
The good news: a working proxy rotator is only a few dozen lines of Python. This tutorial builds one from scratch — starting with a three-line version and ending with a resilient, class-based rotator that retires dead proxies on its own. No frameworks, just the standard library and requests.
What a Proxy Rotator Is and Why You Need One
A proxy rotator is a small component that picks a different proxy for each outgoing request, so the target website sees traffic coming from many IPs instead of one. To the site, your scraper looks like dozens of separate visitors rather than a single aggressive bot.
This directly defeats the most common anti-scraping defense: per-IP rate limiting. When each address makes only a handful of requests, none of them crosses the ban threshold, and your web scraping job keeps running.
Building your own rotator — rather than relying on a black-box endpoint — gives you full control over rotation logic, failure handling, and logging. It's also the best way to understand what premium rotating-proxy services do under the hood.
What You'll Need Before Starting
This tutorial assumes basic Python familiarity and a working Python 3 install. You'll need the requests library, which you can install in one command:
pip install requests
You'll also need a list of proxies. For testing you can use free proxies, but they're slow and unreliable — expect most to be dead. For anything real, use a paid pool; the mini-listicle later in this guide covers solid options. Format each proxy as protocol://user:pass@host:port so it drops straight into requests.
Step 1: Store Your Proxy List
Start by collecting your proxies into a simple Python list. In production you'd load these from a file or environment variable, but a hard-coded list is fine to learn with:
proxies = [
"http://user:pass@host1:port",
"http://user:pass@host2:port",
"http://user:pass@host3:port",
]
Keeping the list separate from your rotation logic means you can swap providers or load hundreds of IPs from a text file without touching the rest of your code. This separation pays off the moment your pool grows.
Step 2: The Simplest Possible Rotator
The most basic rotator just picks a random proxy from the list for each request. Python's random.choice makes this a one-liner:
import random, requests
def get(url):
proxy = random.choice(proxies)
return requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=10)
This works, but random selection can pick the same proxy twice in a row and offers no guarantee of even distribution. For light scraping it's perfectly adequate; for heavier loads you'll want the round-robin approach next.
Step 3: Round-Robin Rotation
Round-robin cycles through every proxy in order before repeating, guaranteeing even usage across the pool. Python's itertools.cycle handles this elegantly:
from itertools import cycle
proxy_pool = cycle(proxies)
def get(url):
proxy = next(proxy_pool)
return requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=10)
Each call to next() advances to the following proxy and loops back to the start automatically. This spreads requests perfectly evenly — ideal when all your proxies are roughly equal in quality and you want predictable distribution.
Step 4: Handling Failures and Retiring Dead Proxies
Real proxies fail. They time out, get banned mid-job, or simply die. A production rotator must catch those failures and stop using bad proxies instead of crashing or retrying the same dead IP forever.
The pattern is to wrap the request in a try/except, and on failure remove the proxy from the active pool and retry with a fresh one. This self-healing behavior is what separates a toy script from a usable tool — track failures and drop any proxy that misbehaves.
It's also worth validating a proxy before trusting it. Our guide on testing proxy speed and success rate shows how to benchmark a pool so you start with healthy IPs.
Step 5: A Complete Class-Based Proxy Rotator
Tying it together, here's a compact class that rotates round-robin, retries on failure, and retires dead proxies automatically. Drop it into any project as a reusable component:
import requests, itertools
class ProxyRotator:
def __init__(self, proxies):
self.proxies = list(proxies)
self.pool = itertools.cycle(self.proxies)
def get(self, url, retries=3):
for _ in range(retries):
proxy = next(self.pool)
try:
r = requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=10)
if r.status_code == 200: return r
except requests.RequestException:
self._retire(proxy)
return None
def _retire(self, proxy):
if proxy in self.proxies: self.proxies.remove(proxy)
self.pool = itertools.cycle(self.proxies)
Use it with rotator = ProxyRotator(proxies) then rotator.get("https://example.com"). It tries up to three proxies per request and permanently drops any that throw an error, keeping your pool healthy over long runs.
Rotation Strategies Compared
Different jobs call for different rotation logic. The table below summarizes the common strategies and when to reach for each.
| Strategy | How It Works | Best For |
|---|---|---|
| Random | Picks any proxy at random | Light, simple scraping |
| Round-robin | Cycles through all in order | Even load across equal proxies |
| Weighted | Favors faster or cleaner IPs | Mixed-quality pools |
| Sticky session | Keeps one IP for a session | Logins, multi-step flows |
Most scrapers start with round-robin and add failure handling, which is exactly what the class above does. Add weighting only once you're tracking per-proxy performance.
Where to Get Reliable Proxies to Rotate
A rotator is only as good as the proxies feeding it. Free lists are mostly dead IPs; a clean paid pool transforms your success rate overnight. These four providers offer rotation-friendly endpoints and developer APIs — compare more in our proxy provider directory.
1Webshare
Best for developers building a rotator on a budget, Webshare offers 10 free proxies and a clean API to pull your list programmatically. It's the easiest way to test rotation logic without spending a cent.
Datacenter speeds are excellent and the dashboard exposes credentials clearly. For learning and small-scale projects, it's the most frictionless starting point.
2Decodo
Great for scaling past the basics, Decodo provides a 115M+ IP pool with built-in rotation and sticky-session endpoints, so you can offload rotation entirely or feed individual IPs to your own rotator. Its 99.99% uptime keeps dead proxies rare.
City-level targeting and per-request metrics make it easy to add weighted rotation later. For mixed workloads, it's a balanced, reliable choice.
3IPRoyal
Ideal for budget-conscious projects, IPRoyal pairs non-expiring residential traffic with simple auth that drops straight into requests. Its 32M+ pool across 195 countries keeps your rotation diverse.
The pay-as-you-go model means unused data carries over between jobs. For freelancers wiring up a first rotator, it's a low-commitment, dependable option.
4Smartproxy
A strong value pick, Smartproxy delivers a 55M+ residential pool with ready-made rotating endpoints and thorough documentation. It bridges the gap between cheap datacenter IPs and premium enterprise tools.
Its rotating gateway can replace your custom logic entirely when you want simplicity. For growing projects that need reliable residential coverage, it's a solid bet.
Testing Your Proxy Rotator
Before turning a rotator loose on a real target, verify it actually rotates. Point it at an IP-echo endpoint and confirm the returned address changes on each request:
for _ in range(5): print(rotator.get("https://httpbin.org/ip").json())
You should see a different IP each time. If the same IP repeats, your rotation logic or proxy pool isn't behaving — check that cycle is advancing and that your provider's endpoint actually rotates. Pair this with the metrics in our proxy testing guide to confirm speed and success rate too.
Common Mistakes to Avoid When Building a Rotator
A rotator that looks correct can still fail in production for a few predictable reasons. Watch for these.
1Forgetting Timeouts
A request without a timeout will hang indefinitely on a dead proxy, freezing your entire scraper. Always set a sensible timeout — 10 seconds is a good default — so failed proxies fail fast and your rotator can move on to the next one.
2Not Removing Dead Proxies
Rotating back to a banned or offline IP wastes requests and slows everything down. Track failures and retire bad proxies from the active pool, as the class example does. A rotator that keeps cycling through dead IPs is barely better than no rotator at all.
3Only Rotating the IP, Not the Fingerprint
A fresh IP paired with the same default user-agent on every request still looks like a bot. Anti-bot systems combine IP and header signals, so rotate realistic user-agents and headers alongside your proxies for traffic that actually blends in.
4Rotating Too Fast for Session-Based Sites
Some targets require you to keep one IP across a login or multi-step flow. Switching IPs mid-session breaks the session and triggers security checks. Use sticky sessions for those cases and reserve fast rotation for stateless, page-by-page scraping.
5Relying on Free Proxy Lists
Free proxies are tempting but overwhelmingly dead, slow, or already blocklisted, and some are outright malicious. Building a rotator on top of them produces constant failures that look like bugs in your code. Start with a clean paid pool and your rotator will simply work.
Best Practices for a Production Proxy Rotator
- Always set timeouts and retries so a single dead proxy never stalls the whole job.
- Retire failing proxies automatically and optionally re-test them later before returning them to the pool.
- Rotate headers and user-agents with the IP — a fresh address with a stale fingerprint still gets flagged.
- Log every request's proxy, status, and timing so you can spot a degrading pool and add weighted rotation later.
- Match proxy type to the target — use residential proxies for protected sites and compare providers in our comparison tool.
Scaling Your Rotator for High-Volume Scraping
The synchronous rotator built above is perfect for moderate jobs, but it makes one request at a time. When you need to scrape thousands of pages quickly, sequential requests become the bottleneck — each one waits for the previous to finish before the next begins.
The fix is concurrency. Swapping requests for an async client like httpx or aiohttp lets you fire dozens of rotated requests in parallel, each using a different proxy from the pool. Your rotation logic barely changes — you still call next() on the cycle — but throughput can jump tenfold because the network waits now overlap instead of stacking up. Pair this with a semaphore that caps how many requests run at once, so you don't overwhelm your proxy pool or the target.
At larger scale, consider moving rotation state into a shared store like Redis. A Redis-backed pool lets multiple worker processes or machines draw from and update the same proxy list — retiring a dead IP in one worker instantly removes it everywhere. This is roughly how distributed scraping frameworks manage proxies across a cluster, and it's a natural next step once a single-process rotator can't keep up.
Whichever path you take, the principles stay identical: rotate evenly, fail fast on dead proxies, retire what doesn't work, and rotate fingerprints alongside IPs. The architecture grows, but the rotator's job never changes. If you'd rather not manage concurrency yourself, a provider's rotating endpoint combined with a tool like our Python web scraping stack handles much of this for you.
Frequently Asked Questions
Conclusion: From Three Lines to a Resilient Tool
Building a proxy rotator in Python is far simpler than it first appears. A working version is three lines with random.choice; a resilient one is a small class that rotates round-robin, retries on failure, and retires dead proxies on its own.
The logic is the easy part — the real lever on your success rate is feeding it a clean, healthy proxy pool and rotating headers alongside IPs. Set timeouts, log everything, and match the proxy type to your target's defenses.
Ready to plug in a real pool? Start with a clean, high-uptime provider from our proxy directory, compare the top options in our side-by-side tool, and go deeper with our companion guide on building a rotating proxy script in Python.