SEO & LocalElite tier

How to Scrape Google Maps at Scale

Google Maps is where local business data lives — names, categories, ratings, review counts, coordinates — and there is no open API that returns it in bulk the way the map does. This guide covers collecting it accurately, city by city, without the feed quietly going half-empty on you.

MTWritten by Marcus Thorne·Reviewed by Liam O'Connor·Updated September 4, 2026
13 min read
Editorial board contrasting a city local pack with map pins against a national-average rank with no pack.

Key takeaways

  • Maps ranks results by the requester's apparent location, so the exit IP's city matters as much as the query itself.
  • The result feed (div[role='feed']) is lazy-loaded — scroll the panel element, not the window, until the 'end of the list' sentinel appears.
  • Dedupe on place_id or the /maps/place/ canonical URL; business names repeat across locations and are not unique.
  • Reviews are user-generated personal data — collect them only with a lawful basis, and prefer aggregates (rating, count) where you can.
  • Bias the viewport with /@lat,lng,zoomz in the URL plus a city-targeted Elite IP so listings match the neighborhood you report on.

On this page

What goes wrong when this scraper fails

A lead-gen team wants every 'HVAC contractor' across 200 US metros; a local-SEO agency needs to see how a client ranks in the Map Pack from each suburb it serves. Both depend on Maps returning what a resident would see. Pull those queries from a single server and Maps either throttles the feed, redirects to a consent wall, or returns a listing set skewed to the datacenter's location — so the 'Chicago' results come back quietly half-populated with the wrong city.

Why this failure mode happens

Maps personalizes ranking by the viewport and the requester's inferred location, then layers Google's standard anti-automation on top: velocity scoring, consent gating (consent.google.com in the EU), and detection of headless or automated browsers. A datacenter IP fails on two fronts at once — it is low-trust and geographically wrong — so even when it is not outright blocked, the listings it returns do not represent any real local searcher.

Four steps to hold a metro pass: pick a city, open a sticky session, keep the same exit, start a new session for the next metro, with a session rail set to lifetime-60.
Sticky defaults to 30 minutes (up to 24 hours) on a rotating-pool IP, not a dedicated address.

Challenges that make this hard to automate

  • Ranking is location-sensitive, so the exit IP's city must match the market, not just the country
  • The feed is JavaScript-rendered and lazy-loaded — raw HTTP returns almost nothing usable
  • Scrolling must continue to the end-of-list sentinel, or you silently truncate the results
  • Reviews are personal data, creating GDPR/CCPA obligations distinct from business facts
  • Business names repeat across locations, so naive name-based dedupe merges distinct places

Approaches that usually fail

  • The official Places API — clean and reliable, but metered, field-limited, and costly across hundreds of metros
  • Datacenter proxies — wrong geography and throttled before a city sweep even finishes
  • A single headless browser on one IP — blocked after a handful of searches
  • Manual collection in Maps — accurate but impossible across 200 cities on a schedule

When residential proxies fix this — and when they cannot

Residential IPs in the target city make each search read as a local user and return the listings residents actually see, while a large rotating pool keeps per-IP velocity low enough to sweep many metros without throttling. City-level exits are precisely what datacenter proxies and VPNs cannot replicate — and for Map Pack work, city precision is the entire point.

Form fields showing place id as the stable join key versus colliding shop names.
Two shops can share a name. place id is the stable join key.

How Aethyn residential proxies help here

Local data collection needs two things at once: geographic precision and enough IP diversity to query hundreds of areas without tripping limits. Aethyn's Elite tier provides both, controlled through the username.

  • Elite city/ISP targeting (-city-) so 'coffee in Chicago' returns Chicago's listings, not a datacenter's
  • A large rotating pool to sweep hundreds of metros while keeping per-IP velocity low
  • Sticky sessions (-session-…-lifetime-) to hold one exit through a single scroll-and-paginate pass
  • 195+ country coverage for international local data from one credential
  • Standard auth that drops straight into Playwright/Puppeteer for the JavaScript feed

How to implement this with residential proxies

  1. 1

    Target the city and bias the viewport

    On Elite, pin country and city in the username so the request originates locally. Then bias the map itself by appending /@lat,lng,zoomz to the search URL — the IP sets who you appear to be, the coordinates set where the map is centered. Together they pin the result set to a real neighborhood.

    cURL
    # City-targeted exit + viewport centered on downtown Chicago (zoom 13)
    curl -x "http://aethyn-XXXXX-country-us-city-chicago:PASSWORD@proxy.aethyn.io:5499" \
      "https://www.google.com/maps/search/coffee+shops/@41.8781,-87.6298,13z"

    Field note: Keep a small lookup of lat/lng centroids per metro you crawl. Pairing the right coordinates with a city-matched IP is what separates a clean local data set from one subtly contaminated by a neighboring region's listings.

  2. 2

    Render and scroll the feed to the end (Playwright)

    Maps is fully JavaScript-driven, so use a headless browser through the residential proxy. The catch most scrapers miss: the results live in a scrollable side panel (div[role='feed']), not the window — you must scroll that element repeatedly until Google appends the 'You've reached the end of the list' sentinel, otherwise you only ever capture the first dozen listings.

    Python (Playwright)
    from playwright.sync_api import sync_playwright
    
    def collect_place_urls(query, lat, lng, max_rounds=40):
        url = f"https://www.google.com/maps/search/{query}/@{lat},{lng},13z"
        with sync_playwright() as p:
            browser = p.chromium.launch(proxy={
                "server": "http://proxy.aethyn.io:5499",
                "username": "aethyn-XXXXX-country-us-city-chicago",
                "password": "PASSWORD",
            })
            page = browser.new_page(locale="en-US")
            page.goto(url)
            feed = page.wait_for_selector("div[role='feed']", timeout=20000)
            for _ in range(max_rounds):
                page.eval_on_selector("div[role='feed']",
                                      "el => el.scrollBy(0, el.scrollHeight)")
                page.wait_for_timeout(1500)
                if page.get_by_text("reached the end of the list").count():
                    break
            links = page.eval_on_selector_all(
                "div[role='feed'] a[href*='/maps/place/']",
                "els => els.map(e => e.href)")
            browser.close()
            return list(dict.fromkeys(links))  # de-dupe, preserve order

    Field note: Cap the scroll rounds and watch for the sentinel — never loop forever. If the feed stops growing well before the sentinel, that is usually a soft throttle, not the genuine end: drop the session, rotate, and resume the metro later.

  3. 3

    Extract structured fields per listing

    Open each place URL (reusing the same sticky session) and read the fields off stable, role-based selectors rather than churned class names. Capture the place_id from the URL as your dedupe key — names like 'Starbucks' appear thousands of times and will collide.

    Python (extraction)
    import re
    
    def parse_place(page):
        name = page.locator("h1").first.inner_text()
        rating = page.locator("div.fontDisplayLarge").first.inner_text()  # e.g. "4.6"
        reviews = page.get_by_text(re.compile(r"\d[\d,]* reviews")).first.inner_text()
        category = page.locator("button[jsaction*='category']").first.inner_text()
        place_id = re.search(r"!1s(0x[0-9a-f:]+)", page.url)
        return {
            "place_id": place_id.group(1) if place_id else None,
            "name": name,
            "rating": float(rating.replace(",", ".")) if rating else None,
            "reviews": int(re.sub(r"[^\d]", "", reviews) or 0),
            "category": category,
            "url": page.url,
        }

    Field note: Selectors like fontDisplayLarge drift over time — wrap each field read in a try/except and log misses. A field that suddenly goes null across every listing is your early-warning that Maps changed its layout, not that every business lost its rating.

  4. 4

    Hold one IP per pass with a sticky session

    A metro sweep is a multi-step flow — search, scroll, then open each place — so pin one exit for that pass with a session id and a short lifetime. Rotate to a fresh exit for the next metro to keep per-IP velocity low across the whole job.

    Username (sticky)
    aethyn-XXXXX-country-us-city-chicago-session-metro042-lifetime-20

    Field note: Derive the session id from the metro (session-metro042) so retries of the same city reuse a coherent identity, while different cities never share one. It makes runs reproducible and easier to debug.

  5. 5

    Collect reviews only with care

    Ratings and review counts are business facts and safe to store. Individual review text and author names are personal data — if you genuinely need them, collect only what your lawful basis covers, minimize what you retain, and honor deletion. For most ranking and lead-gen work, the aggregates are enough.

Best practices that keep scrapers reliable

  • Pair a city-matched Elite IP with a /@lat,lng viewport for true local results
  • Scroll the feed element to the end-of-list sentinel — don't stop at the first screen
  • Dedupe on place_id, never on business name
  • Read fields off role-based selectors and wrap each in try/except for layout drift
  • Re-crawl on a cadence and store deltas; ratings move slowly, listings churn
  • Treat ratings/counts as facts but review text/authors as personal data

Common mistakes that burn proxy budget

  • Scrolling the window instead of the feed panel, capturing only the first dozen listings
  • Stopping scroll before the sentinel and silently truncating each metro
  • Deduping on name, merging 'Starbucks' locations into one record
  • Querying a city with a datacenter IP and trusting the skewed listing set
  • Hoarding review text with no lawful basis or retention limit
  • Treating a stalled feed as the genuine end of results rather than a soft throttle

Common questions about this scraper problem

Is there an API instead of scraping Maps?
The Google Places API returns structured place data, but it is metered, exposes a limited field set, and gets expensive across hundreds of metros or for full ranking snapshots. Many teams scrape the public map specifically to capture the live, location-ranked result order the API does not give them.
Is scraping Google Maps legal?
Public business listings are widely collected for research and lead generation, but it can conflict with Google's Terms of Service, and reviews are personal data. Limit collection to public business facts, apply privacy law to anything user-generated, and consult counsel for your use case.
Why do I only get about a dozen results?
You are almost certainly scrolling the page window instead of the results panel. Maps lazy-loads listings into div[role='feed']; you must scroll that element repeatedly until the 'reached the end of the list' sentinel appears.
Do I really need a headless browser?
For most Maps work, yes. The feed and place panels are JavaScript-rendered and lazy-loaded, so Playwright or Puppeteer through a residential proxy is far more reliable than raw HTTP.
How do I get city-level accuracy?
Use the Elite tier with -city- in the username and center the map with /@lat,lng,zoomz. The IP controls who you appear to be; the coordinates control where the map looks.
Why are my results inconsistent between runs?
Maps ranking shifts with viewport, apparent location, and time. Pin a consistent city and coordinates, hold one sticky session per metro pass, and re-crawl on a schedule to capture genuine changes rather than noise.
Rotating or sticky sessions for Maps?
Use a sticky session for a single metro pass (search → scroll → open places) so the flow stays coherent, then rotate to a fresh exit for the next metro to keep per-IP velocity low.
What is the best dedupe key?
The place_id embedded in the /maps/place/ URL. Business names collide constantly across locations, so name-based dedupe corrupts your dataset.
Which tier should I use?
Elite — Maps is high-security, and only the Elite tier offers the city/ISP precision that local ranking data actually requires.
Share this article

Solve this with Aethyn residential proxies

Route your first request through rotating residential IPs in minutes, with transparent per-byte pricing.

All solutions