TravelElite tier

How to Scrape Booking.com

On Booking.com a hotel has no fixed price — it has a rate for specific dates, occupancy, currency, and length of stay. This guide covers collecting hotel availability and rates accurately, with the date/occupancy parameters that define a real quote, while getting past one of travel's tougher anti-bot setups.

SLWritten by Sophie Laurent·Reviewed by Elena Vance·Updated September 4, 2026
12 min read
Editorial board contrasting a hotel card with one fake price against a rate defined by dates, occupancy, rooms, and currency.

Key takeaways

  • A rate only exists for a query: check-in, check-out, occupancy, rooms, and currency all change the number.
  • Pin the currency explicitly (selected_currency) — don't let it vary by IP, or your price series mixes currencies.
  • Results paginate via an offset parameter; walk it until results stop, and respect the per-search result count.
  • Key on the Booking hotel ID; capture room type and cancellation/board terms alongside the rate.
  • Reviews and reviewer data are personal data — collect property rates and aggregate scores, not individuals.

On this page

What goes wrong when this scraper fails

Hotels, OTAs, and travel-intelligence teams need competitive room rates and availability across markets and dates. Booking.com makes this genuinely hard: the price depends entirely on the search (dates, guests, rooms, currency, length of stay), so a snapshot without those parameters is meaningless; rates and even which deals show can vary by currency and apparent geography; and the site runs strong bot detection that blocks datacenter IPs and abnormal patterns. Collected carelessly, the data is a mix of currencies and date scenarios that no analyst can trust.

Why this failure mode happens

Booking.com computes rates in real time from the property's inventory and your exact search parameters, and it localizes currency and some offers to the visitor. It defends its pages with layered bot management (challenges and reCAPTCHA) that scores IP reputation, fingerprint, and behavior. So an automated request from a datacenter IP is both likely to be challenged and, if it succeeds, likely to reflect the wrong currency or a non-comparable date scenario.

Form pinning selected_currency so a hotel rate series does not mix units by exit IP.
Pin selected_currency. If currency follows the exit IP, the price series mixes units.

Challenges that make this hard to automate

  • Rates that only exist relative to dates, occupancy, rooms, and length of stay
  • Currency and offers that shift with apparent geography if not pinned
  • Heavy bot detection and challenges on a high-value OTA
  • Offset-based pagination and per-search result limits
  • Reviews and reviewer profiles that are personal data

Approaches that usually fail

  • Datacenter proxies — challenged quickly on a defended OTA
  • Collecting 'a price' without fixing dates/occupancy — non-comparable noise
  • Letting currency follow the IP — silently mixed price series
  • Manual rate shopping — accurate but unscalable across markets and dates

When residential proxies fix this — and when they cannot

Residential IPs in the target market read as real travelers, clear the bot detection that blocks datacenter ranges, and (with currency pinned) return the rates a local guest would be quoted. A large rotating pool lets you run the many date/market combinations a rate-shopping study needs without any single IP tripping velocity limits.

Offset pagination rail walking Booking.com hotel results until the page is empty.
Walk the offset until hotels stop. Capture room type and cancellation terms with the rate.

How Aethyn residential proxies help here

Booking.com is a high-security, geo- and currency-sensitive target. Aethyn supplies the reputation and IP diversity it demands, controlled through the username.

  • Elite high-trust residential IPs to get past Booking's bot management
  • Country targeting so availability and localized offers match the market
  • A large rotating pool for many date × market rate lookups
  • Sticky sessions to keep a search → property-detail flow coherent
  • Per-byte metering so recurring rate shopping stays cost-predictable

How to implement this with residential proxies

  1. 1

    Define the search: dates, occupancy, rooms, currency

    A Booking rate is only meaningful with the full query. Set check-in/check-out, group/adults and rooms, and crucially pin the currency with selected_currency so it doesn't follow the IP. Standardize a scenario (e.g. 2 adults, 1 room, 2 nights, 30 days out, EUR) and apply it consistently.

    cURL
    curl -x "http://aethyn-XXXXX-country-fr:PASSWORD@proxy.aethyn.io:5499" \
      "https://www.booking.com/searchresults.html?ss=Paris&checkin=2026-08-10&checkout=2026-08-12&group_adults=2&no_rooms=1&selected_currency=EUR"

    Field note: Always set selected_currency explicitly. If you let currency default to the exit IP's country, you'll silently build a price series that mixes EUR, USD, and GBP — and no amount of later cleaning fully untangles it.

  2. 2

    Render and read the structured rate data

    Drive a headless browser through the residential proxy and wait for the results to render, since rates load dynamically. Extract per-property: hotel ID, name, room type, the rate for your scenario, board/cancellation terms, and the aggregate review score. Prefer embedded data attributes over volatile class names.

    Python (Playwright)
    from playwright.sync_api import sync_playwright
    
    def search(url):
        with sync_playwright() as p:
            b = p.chromium.launch(proxy={
                "server": "http://proxy.aethyn.io:5499",
                "username": "aethyn-XXXXX-country-fr",
                "password": "PASSWORD",
            })
            page = b.new_page(locale="fr-FR")
            page.goto(url, wait_until="networkidle")
            cards = page.query_selector_all("[data-testid='property-card']")
            rows = []
            for c in cards:
                name = c.query_selector("[data-testid='title']")
                price = c.query_selector("[data-testid='price-and-discounted-price']")
                rows.append({"name": name.inner_text() if name else None,
                             "price": price.inner_text() if price else None})
            b.close()
            return rows

    Field note: data-testid attributes are far more stable than Booking's hashed class names. Anchor parsing on them and guard each field — when a testid disappears across the board, that's your signal Booking changed the markup, not that hotels lost prices.

  3. 3

    Paginate through the full result set

    Booking paginates with an offset parameter (in steps matching the page size). Increment offset and keep collecting until a page returns no new properties, deduping on hotel ID. Respect that a single search only surfaces so many results — narrow by area or filters for dense markets.

    Python (offset pagination)
    def all_results(base_url, fetch, step=25, max_offset=1000):
        seen = {}
        for offset in range(0, max_offset, step):
            rows = fetch(f"{base_url}&offset={offset}")   # one residential request each
            if not rows:
                break
            for r in rows:
                seen[r["hotel_id"]] = r
        return list(seen.values())

    Field note: For big destinations, slice the search (by district, star rating, or property type) instead of paginating endlessly. Narrower searches return more complete, less truncated result sets than one broad query.

  4. 4

    Capture rate terms, not just the number

    Two rates at the same price aren't equal if one is non-refundable and room-only and the other is free-cancellation with breakfast. Store board type, cancellation policy, and room type with every rate so comparisons are apples-to-apples and rate-parity analysis is valid.

    Field note: For rate-parity work, also record whether a 'Genius'/member discount applied. A price gap between two sites is often a loyalty discount, not a true parity violation.

  5. 5

    Schedule, rotate, and trend responsibly

    Run your scenarios on a cadence through rotating residential IPs, store (hotel_id, scenario, currency, rate, terms, timestamp), and trend rates and availability. Collect property-level review scores rather than individual reviews, and back off on any challenge.

Best practices that keep scrapers reliable

  • Fix dates, occupancy, rooms, and currency before collecting any rate
  • Pin selected_currency explicitly so series don't mix currencies
  • Anchor parsing on data-testid, not hashed class names
  • Paginate by offset until empty, deduping on hotel ID
  • Store board/cancellation/room-type terms with every rate
  • Collect aggregate review scores, not individual reviewer data

Common mistakes that burn proxy budget

  • Collecting a rate without dates/occupancy, producing non-comparable data
  • Letting currency follow the IP and mixing currencies in one series
  • Comparing a non-refundable room-only rate to a flexible breakfast rate
  • Stopping pagination early and truncating dense markets
  • Scraping individual reviews/reviewer profiles with no lawful basis
  • Running a defended OTA from datacenter IPs and getting challenged

Common questions about this scraper problem

Why doesn't a hotel have a single price on Booking.com?
Rates are computed per search: check-in/check-out, occupancy, number of rooms, length of stay, and currency all change the number, as do cancellation and board options. To get comparable data you must collect for a fixed, documented scenario rather than 'the price'.
Why are my prices in mixed currencies?
Because Booking localizes currency to the visitor unless you pin it. Always set selected_currency in the URL so every rate is in the same currency; otherwise the exit IP's geography silently changes the currency and corrupts your price series.
How do I extract rates reliably from the page?
Render the search with a headless browser through a residential proxy (rates load dynamically) and anchor your parser on data-testid attributes like property-card and price-and-discounted-price, which are far more stable than Booking's hashed CSS class names.
How does pagination work?
Booking uses an offset query parameter in steps matching the page size. Increment offset until a page returns no new properties, deduping on hotel ID. For large destinations, slice the search by area or filters to avoid truncated result sets.
What should I capture besides the price?
Room type, board (e.g. room-only vs breakfast), and cancellation policy — and whether a Genius/member discount applied. Two equal prices aren't equal offers, and these fields are what make rate comparisons and parity checks valid.
Is scraping Booking.com legal?
Collecting public rate and availability data is common for rate intelligence, but Booking's Terms restrict automated access and reviews contain personal data. Limit collection to public property data, avoid personal/reviewer data, and consult counsel for your jurisdiction and use case.
Which tier should I use?
Elite. Booking.com is a high-value OTA with heavy bot detection, so the higher-reputation residential pool plus country targeting (for correct availability and localized offers) gives the most reliable collection.
How do I keep recurring rate shopping affordable?
Aethyn bills per byte, so limit scenarios to the ones you actually analyze, slice searches to avoid over-paginating, and re-collect only the date ranges and markets that matter on a sensible cadence.
Share this article

Solve this with Aethyn residential proxies

Route your first request through rotating residential IPs in minutes, with transparent per-byte pricing.

All solutions