Web ScrapingPremium tier

How to Rotate Residential IPs Automatically

Most teams over-build IP rotation — IP lists, health checks, ban tracking — to solve a problem a managed network already handles. The real skill is knowing when you want a fresh IP per request and when you need the same one, and expressing that in a single line. This guide covers both, cleanly.

TMWritten by Thomas Müller·Reviewed by Liam O'Connor·Updated September 4, 2026
11 min read
Editorial board contrasting a hand-rolled IP list with rotation expressed as username flags on one endpoint.

Key takeaways

  • With a managed network, rotation is a username choice, not infrastructure you maintain.
  • Default to a fresh IP per request for stateless work; use a sticky session only when state must persist.
  • A sticky session is just a session id plus a lifetime in the username — different ids get different IPs.
  • Give every concurrent worker its own session id, or they'll fight over one shared exit IP.
  • Rotation controls velocity and reputation — it does not fix fingerprint, so pair it with header/TLS hygiene.

On this page

What goes wrong when this scraper fails

It's tempting to write a 'rotation manager': fetch a list of proxies, health-check them, track which ones got banned, round-robin the survivors. With a real residential network this is wasted effort — the pool is millions of IPs the provider already health-checks and cycles. The genuine decisions are narrower and more important: per-request rotation versus a sticky session, how long to pin a session, and how to make concurrent workers rotate independently without trampling each other.

Why this failure mode happens

Hand-rolled rotation is fragile because IP lists go stale, banned IPs linger, and coordinating 'who uses which IP' across workers is a distributed-systems problem you don't want. At the same time, blind always-rotate logic breaks the workflows that genuinely need a stable identity — logins, multi-page pagination, carts — because the session's cookies suddenly arrive from a new IP and the site invalidates them.

Two panes: a fresh IP per request versus a sticky session for login and pagination.
Sticky is a session id plus a lifetime in the username, not a dedicated IP.

Challenges that make this hard to automate

  • Choosing fresh-per-request vs a stable IP for each kind of job
  • Avoiding stale or banned IPs that manual lists inevitably accumulate
  • Preserving session continuity for logins and multi-step flows
  • Coordinating independent rotation across many concurrent workers
  • Remembering that rotation alone doesn't defeat fingerprint-based detection

Approaches that usually fail

  • Maintaining your own IP list — endless upkeep, and bans you discover the hard way
  • Round-robin over a few proxies — a small pool that gets flagged together fast
  • Custom ban-tracking and health-check logic — complex, stateful, and error-prone
  • A network with no stickiness option — quietly breaks every login and cart flow

When residential proxies fix this — and when they cannot

A managed residential network rotates across a huge, continuously-refreshed pool for you. You stop maintaining anything and simply declare intent: the base username for a fresh IP per request, or a session id plus lifetime when a flow needs continuity. The network handles pool health, geography, and assignment.

Three workers: unique session ids versus a shared session that collides on one exit.
Rotation does not fix TLS fingerprint. Pair it with header and TLS hygiene.

How Aethyn residential proxies help here

Aethyn expresses rotation entirely through the username, so one endpoint gives you either a fresh IP per request or a pinned session — no second gateway, no client-side IP bookkeeping.

  • Per-request rotation by default — a fresh residential IP on every call
  • Sticky sessions via -session-<id> with -lifetime-<minutes> (up to 1440)
  • Independent sessions: different session ids map to different stable IPs
  • Country (and Elite city) targeting composes with either mode in one string
  • Identical syntax across HTTP/HTTPS and SOCKS5, Premium and Elite

How to implement this with residential proxies

  1. 1

    Rotate on every request (the default)

    For stateless scraping — independent page fetches with no shared login — use the base username. Every request is assigned a fresh residential IP automatically, which keeps per-IP velocity low without you tracking anything. Verify it with two calls to an IP-echo endpoint.

    cURL (rotating)
    curl -x "http://aethyn-XXXXX:PASSWORD@proxy.aethyn.io:2099" https://httpbin.org/ip
    curl -x "http://aethyn-XXXXX:PASSWORD@proxy.aethyn.io:2099" https://httpbin.org/ip
    # Two different exit IPs

    Field note: This is the right default for ~80% of scraping. Reach for stickiness only when you can name the specific reason the same IP must persist — if you can't, you don't need it.

  2. 2

    Pin one IP with a sticky session

    When a flow must look like one continuous user — log in, then load several authenticated pages — add a session id and a lifetime in minutes. Every request carrying the same session string lands on the same exit IP until the lifetime expires.

    cURL (sticky)
    curl -x "http://aethyn-XXXXX-session-job7-lifetime-10:PASSWORD@proxy.aethyn.io:2099" https://httpbin.org/ip
    curl -x "http://aethyn-XXXXX-session-job7-lifetime-10:PASSWORD@proxy.aethyn.io:2099" https://httpbin.org/ip
    # Same exit IP for ~10 minutes

    Field note: Choose the shortest lifetime that covers the flow. A 30-minute pin for a 90-second login is just a slow way to over-use one IP — exactly the velocity concentration rotation exists to avoid.

  3. 3

    Give each concurrent worker its own session

    The classic parallelism bug is many workers sharing one sticky session — they all pile onto a single IP and get it banned. Derive a unique session id per worker (or per logical task) so each gets an independent, stable exit. Compose country targeting into the same string when you need it.

    Python (per-worker session)
    def proxy_for_worker(worker_id, country="us"):
        user = f"aethyn-XXXXX-country-{country}-session-w{worker_id}-lifetime-15"
        return f"http://{user}:PASSWORD@proxy.aethyn.io:2099"
    
    # Worker 1 and worker 2 get different stable IPs:
    proxy_for_worker(1)   # ...-session-w1-...
    proxy_for_worker(2)   # ...-session-w2-...

    Field note: Make session ids deterministic from the unit of work (session-w{id} or session-{account_id}), not random per request. Deterministic ids make retries reuse the same identity and make runs reproducible when you're debugging a ban.

  4. 4

    Use rotation per request in an async pool

    With per-request rotation you don't pre-allocate IPs — you just point your async client at the rotating endpoint and let every coroutine pick up a fresh exit. This is why rotation pairs so naturally with high concurrency: there's no shared IP state to coordinate.

    Python (httpx async, rotating)
    import asyncio, httpx
    
    ROTATING = "http://aethyn-XXXXX:PASSWORD@proxy.aethyn.io:2099"
    
    async def fetch(client, url):
        r = await client.get(url, timeout=30)   # fresh IP per request
        return r.status_code
    
    async def main(urls):
        async with httpx.AsyncClient(proxy=ROTATING) as client:
            return await asyncio.gather(*(fetch(client, u) for u in urls),
                                        return_exceptions=True)

    Field note: Rotation fixes reputation and velocity, not fingerprint. If you're still blocked after rotating cleanly, the problem is upstream of the IP — check headers and TLS fingerprint (see the IP-blocking guide).

Best practices that keep scrapers reliable

  • Let the network manage the pool — don't build IP lists or health checks
  • Default to per-request rotation; justify every sticky session
  • Set the shortest lifetime that covers the flow
  • Derive a unique, deterministic session id per worker or task
  • Compose -country/-city into the same username instead of separate gateways
  • Remember rotation is one layer — keep headers and fingerprint clean too

Common mistakes that burn proxy budget

  • Re-implementing rotation the managed network already does
  • Sharing one sticky session across unrelated parallel workers, banning the IP
  • Pinning a long lifetime for a short flow and concentrating velocity
  • Using random per-request session ids, so retries can't reuse an identity
  • Rotating mid-login and invalidating the session's cookies
  • Assuming rotation alone defeats fingerprint-based detection

Common questions about this scraper problem

Do I need to build my own IP rotation logic?
No — that's the main takeaway. A managed residential network already maintains, health-checks, and cycles a huge pool. You just choose per-request rotation or a sticky session via the username; building IP lists and ban tracking on top is wasted, fragile effort.
How does automatic rotation actually work?
Use the base username and the network assigns a fresh residential IP to each request. There's no list to fetch, no health-checking, and no coordination on your side — the rotation happens at the endpoint.
How do I keep the same IP across several requests?
Add a session id and lifetime to the username, e.g. -session-job7-lifetime-10. Every request with that exact session string uses the same exit IP until the lifetime (30 minutes by default, up to 24 hours) expires.
When should I use sticky versus rotating?
Rotate per request for stateless work to keep velocity low and concurrency simple. Use a sticky session only for flows that need one identity — login, multi-page pagination, carts, or paging a single query.
How do I rotate across many concurrent workers?
Give each worker a unique, deterministic session id (e.g. session-w{id}). They'll each get an independent stable IP. Sharing one session across workers is the classic mistake that bans the shared IP.
Can I rotate and target a country at the same time?
Yes. Combine -country-XX (and -city- on Elite) with either the base username for rotation or a session id for stickiness — it's all one string.
What's the maximum sticky session length?
24 hours (1440 minutes); the default is 30 minutes. Set it explicitly with -lifetime-<minutes>, and prefer the shortest value that covers your flow.
I'm rotating cleanly but still blocked — why?
Rotation only addresses IP reputation and velocity. If you're still flagged, the cause is usually fingerprint or headers — a non-browser TLS/JA3 signature or inconsistent header set. See the guide on preventing IP blocking for the other layers.
Share this article

Solve this with Aethyn residential proxies

Route your first request through rotating residential IPs in minutes, with transparent per-byte pricing.

All solutions