SEO & LocalPremium tier

How to Track Search Rankings Across Countries

Two people searching the same keyword in different cities see different rankings — so 'where do we rank?' has no single answer until you pin a location. This guide covers building rank tracking that measures the position a real local searcher sees, consistently, across countries and cities, without poisoning the data with personalization or blocks.

MTWritten by Marcus Thorne·Reviewed by Elena Vance·Updated September 4, 2026
12 min read
Editorial board contrasting a country-level SERP order with a city-level order that includes a local pack.

Key takeaways

  • Rank is location-specific: report it per market, never as one global number.
  • Neutralize personalization (no logged-in cookies, pws=0) so you measure the market, not your own history.
  • The exit IP geography must agree with gl/hl — and with the city, for local-intent terms.
  • Decide what counts as position 1: ads, the local pack, and AI Overviews push organic results down and must be handled explicitly.
  • Store time-stamped positions plus the SERP features present, so a 'drop' caused by a new AI Overview isn't mistaken for a ranking loss.

On this page

What goes wrong when this scraper fails

An SEO team reports rankings to clients across a dozen countries and, for local businesses, dozens of cities. Checking from the office gives a single personalized, location-biased view that matches no client's actual market. Worse, the SERP itself has changed: ads, the local pack, 'People also ask', and AI Overviews now occupy the space organic results used to, so a naive 'position 3' can mean wildly different visibility than it did two years ago. Accurate tracking has to control location, neutralize personalization, and account for what the modern SERP actually looks like.

Why this failure mode happens

Search results are localized by the requester's IP and the gl/hl parameters, and personalized by account history and prior searches. A central server sees exactly one biased view. Layer on velocity-based throttling for repeated automated queries from datacenter IPs, and the data is both unrepresentative and unreliable. And because Google increasingly injects SERP features above the organic block, position numbers are only meaningful if you also record what surrounds them.

Form matching hl, gl, and uule locale parameters to the proxy exit country and city.
A US exit with a DE locale still renders a mixed SERP. Match both.

Challenges that make this hard to automate

  • Rankings that legitimately differ by country and, for local intent, by city
  • Personalization (history, login) biasing any single-location check
  • Velocity throttling that corrupts repeated automated queries
  • Keeping IP geography, gl/hl, and city all consistent
  • Counting position correctly when ads, local packs, and AI Overviews intrude

Approaches that usually fail

  • Manual incognito checks — still biased by your own IP and impossible to schedule
  • Datacenter proxies — throttled and geographically wrong for the market
  • Generic rank tools — convenient but often coarse on geo and opaque on method
  • Trusting 'incognito' to remove bias — it hides history but not IP-based localization

When residential proxies fix this — and when they cannot

Querying from residential IPs in each target market returns the localized rankings real users in that market see, and rotating across the pool keeps per-query velocity low so scheduled tracking stays clean and unblocked. For local-intent terms, city-level exits capture the neighborhood-by-neighborhood variation that national tracking simply cannot.

Two SERP panes paginating with start offsets instead of requesting num=100.
num=100 is a block magnet. Walk start offsets instead.

How Aethyn residential proxies help here

Rank tracking is recurring, geo-specific, and volume-sensitive. Aethyn maps country (and Elite city) targeting onto a single rotating endpoint so one tracker sweeps every market you report on.

  • Country targeting aligned to gl/hl for accurate national rankings
  • Elite city/ISP targeting for local-pack and local-intent terms
  • 195+ countries for full international rank coverage
  • Per-request rotation to keep query velocity low across large keyword sets
  • Premium pricing that keeps daily/weekly scheduled tracking affordable

How to implement this with residential proxies

  1. 1

    Define the keyword × market matrix

    Map each keyword to the countries — and, for local intent, the cities — you report on, with the matching language. This matrix is the unit of tracking: 'best running shoes' in (de, de-DE) is a different measurement from the same term in (us, en-US), and you'll store and trend them separately.

    Field note: Tag each row with search intent. Local-intent terms ('dentist', 'gym near me') need city-level tracking; informational terms usually only need country. Tracking everything at city level is expensive and noisy for terms where the city doesn't change the result.

  2. 2

    Query each market with a neutralized, locale-matched request

    Set the proxy country and the engine's gl/hl to the same market, and neutralize personalization: send no logged-in cookies and add pws=0 to drop personalized results. The IP says where you are, gl/hl says which market's index to use, and pws=0 says 'don't bias this by anyone's history'.

    Python (requests)
    import requests
    
    def serp(keyword, country, hl, start=0):
        proxy = f"http://aethyn-XXXXX-country-{country}:PASSWORD@proxy.aethyn.io:2099"
        proxies = {"http": proxy, "https": proxy}
        params = {"q": keyword, "gl": country, "hl": hl, "pws": 0, "start": start}
        headers = {"Accept-Language": f"{hl},{hl};q=0.9",
                   "User-Agent": "Mozilla/5.0 ... Chrome/124.0 Safari/537.36"}
        # fresh client => no carried-over cookies/personalization
        return requests.get("https://www.google.com/search",
                            params=params, headers=headers, proxies=proxies, timeout=30)
    
    r = serp("residential proxies", "de", "de-DE")
    print(r.status_code)

    Field note: Use a fresh, cookie-less request per check. A persisted session quietly accumulates personalization signals, so your 'market ranking' slowly drifts toward whatever this particular session has been doing.

  3. 3

    Count position the way the SERP actually renders

    Decide and document your position model before parsing: are ads, the local pack, and AI Overviews counted, or only the classic organic (#rso a > h3) results? Find your target domain in that ordered list and record the rank — plus the SERP features present — so the number is interpretable later.

    Python (find position)
    from urllib.parse import urlparse
    from bs4 import BeautifulSoup
    
    def find_position(html, target_domain):
        soup = BeautifulSoup(html, "html.parser")
        rso = soup.select_one("div#search div#rso")
        if rso is None:
            raise BlockedError("No #rso — soft block or consent wall")
        for i, h3 in enumerate(rso.select("a > h3"), start=1):
            href = h3.find_parent("a").get("href", "")
            if urlparse(href).netloc.endswith(target_domain):
                return i                 # organic position
        return None                      # not in this page of results

    Field note: Record the SERP features alongside the position (ai_overview, local_pack, ads_count). A page can hold position 1 organically and still lose clicks to an AI Overview above it — tracking only the number hides that story from your reporting.

  4. 4

    Schedule, rotate, and trend

    Run on a daily or weekly cadence, rotate IPs per query to keep velocity low, paginate with start=10,20 for deeper positions, and back off on any challenge. Persist (keyword, market, position, features, timestamp) so reports show movement — and so a drop caused by a new SERP feature is distinguishable from an actual ranking loss.

Best practices that keep scrapers reliable

  • Track per market (country, and city for local intent), never as one global number
  • Neutralize personalization: cookie-less requests plus pws=0
  • Keep exit IP, gl/hl, and city all aligned
  • Document and apply a consistent position model (what counts as rank 1)
  • Record SERP features alongside position, not just the number
  • Rotate per query, store time-stamped history, and back off on challenges

Common mistakes that burn proxy budget

  • Reporting one location's rankings as if they were global
  • Mismatching IP country and gl/hl, measuring a market no user sees
  • Carrying cookies/personalization across checks and drifting the data
  • Counting position inconsistently across runs so trends are meaningless
  • Ignoring AI Overviews/local packs and misreading lost clicks as lost rank
  • Querying too fast and throttling the very pipeline you depend on

Common questions about this scraper problem

Why do rankings differ by location?
Search engines localize by the requester's IP and gl/hl parameters and personalize by account history. To report accurate rankings for a market you must query as a cookie-less user located in it — which residential IPs in that geography provide.
How do I remove personalization from results?
Send requests without logged-in cookies (a fresh client per check) and add pws=0 to disable personalized results. Combined with the correct IP geography and gl/hl, that gives you the market's view rather than one shaped by a session's history.
How do I count position now that SERPs have ads and AI Overviews?
Pick a position model and apply it consistently: typically track classic organic results separately and record SERP features (AI Overview, local pack, ad count) alongside the rank. That way a visibility drop caused by a new feature is distinguishable from an actual ranking change.
Do I need city-level tracking?
For local-intent terms (local services, 'near me'), yes — use the Elite tier to target cities, because rankings shift block by block. For national, informational terms, country targeting on Premium is usually sufficient.
How often should I track rankings?
Daily or weekly is typical. Rotate IPs per query and keep velocity low so scheduled runs stay unblocked and the data stays comparable over time.
How do I keep results consistent between runs?
Always pair the proxy country with gl/hl (and the city where relevant), neutralize personalization the same way every time, and use one fixed position model. Consistency of method is what makes the trend line trustworthy.
Which tier should I use?
Premium is cost-efficient for most rank tracking. Use Elite for city-level local tracking or for stubborn high-security queries that Premium struggles with.
Is automated rank tracking against search engine terms?
Automated querying can conflict with search engines' Terms of Service. Collect only public ranking data, keep volume reasonable, and consult counsel for your specific use case and jurisdiction.
Share this article

Solve this with Aethyn residential proxies

Route your first request through rotating residential IPs in minutes, with transparent per-byte pricing.

All solutions