Solutions

Real-world proxy solutions, explained.

Step-by-step, code-first guides for solving the data-collection problems teams actually face — scraping without bans, tracking SERPs by country, monitoring prices, verifying ads, and testing geo-restricted experiences with residential proxies.

Showing 1–12 of 57 solutions

Official Data API card with a client id and public listing rows, contrasted with sign-in, vote, and message actions marked out of scope

How to Collect Public Reddit Posts

Premium

Collecting public Reddit posts means reading listings and comments a logged-out visitor can already see, through Reddit's official Data API first. A residential proxy changes the exit those reads leave from. It does not replace the OAuth token, and it does not make a sign-in, a vote, or a post acceptable.

Web Scraping
Editorial board contrasting a green n8n editor run that stored euro prices with a scheduled run of the same workflow that stored dollar prices from another country.

How to Use Residential Proxies with n8n

Premium

Configure n8n's HTTP Request node with a residential proxy URL, verify the exit country, then store a classified catalog row. Options → Proxy is a full HTTP URL (scheme, username, password, host, port) — not a VPN and not SOCKS5 in that field. HTTP 200 is not success: classify populated only when you extracted at least one title-and-price record, and stop the catalog GET when the verified country does not match. This page is the how-to. Why a green editor run can still be the wrong market is the companion blog. Examples follow n8n HTTP Request docs (Proxy option, Include Response Headers and Status). They were not executed end-to-end on n8n Cloud for this page.

Automation
Editorial board showing Cloudflare Error 1020 decided at the edge so the scraper never reaches origin.

How to Scrape Sites Protected by Cloudflare

Elite

Cloudflare is not one check — it is a stack of five layers that score your request before HTML reaches your parser. This architecture guide maps each layer to what it measures, which tools address it, and how residential proxies fit without pretending a proxy alone solves JavaScript challenges.

Web Scraping Core
Editorial board contrasting a classic Google blue-link SERP with a Shopping offer grid of merchant, price, and shipping chips.

How to Scrape Google Shopping Results Without Getting Blocked

Elite

Google Shopping results are merchant offers: who sells a product, at what price, in which currency, with what shipping or store-pickup promise. They are not organic search rankings. A classic Google Search SERP is a ranked list of pages. Collectors usually read it from the #rso container. Google Shopping is a separate surface — historically tbm=shop, increasingly the udm=28 Shopping UI — that hydrates an offer grid after JavaScript runs. Price-intelligence and technical SEO teams often discover the difference the hard way. A Python requests GET to google.com/search can return HTTP 200 and still contain a consent wall, a /sorry/ challenge, an empty carousel, or a classic SERP with no offer cards. Dashboards stay green. The stored “lowest price” describes nobody. Reliable collection is not a promise that you will never be blocked. It is a pipeline: pin a geo-matched residential exit, render the public Shopping page in a real browser, classify the document before you write a price, and keep country-catalog rows separate from city-local inventory. This guide is that collector. It does not bypass Google, solve CAPTCHAs, or unlock a merchant feed. For why the same SKU shows different merchants across metros, use the city-price-divergence protocol after this pipeline is in place.

SERP & Search
Editorial board contrasting a Google AI Overview block above organic results with an AI Mode conversation and numbered citation pills.

How to Collect Google AI Mode Answers and Citations

Elite

Google AI Mode is a rendered conversational surface, not the AI Overview block above blue links and not a public API. Teams that treat it as another SERP HTML fetch poison GEO dashboards. This guide collects answers and citations with Playwright on geo-matched Elite residential exits, then keys each record so empty is never stored as not cited.

SERP & Search
Editorial board contrasting a Google /sorry/ interstitial with a real SERP that contains #rso result rows.

How to Scrape Google Search Results Without Getting Blocked

Elite

Google has no official organic-results API, so rank trackers and SERP tools have to query the public pages directly — and Google is one of the most aggressive anti-automation surfaces on the web. Here is how to collect accurate, localized SERP data at volume without drowning in 'unusual traffic' pages.

SERP & Search
Editorial board contrasting a city local pack with map pins against a national-average rank with no pack.

How to Scrape Google Maps at Scale

Elite

Google Maps is where local business data lives — names, categories, ratings, review counts, coordinates — and there is no open API that returns it in bulk the way the map does. This guide covers collecting it accurately, city by city, without the feed quietly going half-empty on you.

SEO & Local
Editorial board contrasting a single IP lever with a four-layer block score of reputation, velocity, TLS, and behavior.

How to Prevent IP Blocking During Web Scraping

Elite

IP bans are the single most common reason scrapers fail — but 'the IP got banned' is usually shorthand for several different detection signals firing at once. This guide breaks down exactly what modern anti-bot systems measure and how to design a pipeline that reads as ordinary traffic.

Web Scraping
Editorial board contrasting a synchronous for-loop with bounded asyncio behind a semaphore.

How to Scale Python Web Scrapers

Premium

A scraper that flies through 500 pages can fall apart at 5 million — too slow, out of memory, or blocked within minutes. Scaling is the art of doing concurrency, reliability, and rotation at the same time. This guide shows the production patterns that get you there in Python.

Web Scraping
Editorial board contrasting an empty HTML root shell with a fully rendered JavaScript listing page.

How to Scrape JavaScript-Rendered Websites

Elite

View-source shows an empty <div id="root"></div>, but the browser shows a full page of data. That gap — content rendered client-side by JavaScript — is why requests.get returns nothing useful. This guide covers capturing fully rendered pages with headless browsers and residential proxies, plus the faster path most people miss.

Web Scraping
Editorial board contrasting a hand-rolled IP list with rotation expressed as username flags on one endpoint.

How to Rotate Residential IPs Automatically

Premium

Most teams over-build IP rotation — IP lists, health checks, ban tracking — to solve a problem a managed network already handles. The real skill is knowing when you want a fresh IP per request and when you need the same one, and expressing that in a single line. This guide covers both, cleanly.

Web Scraping
Editorial board contrasting a cloaked clean page from an office IP with the real Berlin creative seen through a residential exit.

How to Verify Digital Ads Across Countries

Premium

The same campaign serves different creative in Berlin, London, and São Paulo — and ad fraud thrives precisely in the geos no one is watching. This guide shows how to verify creative, placement, and landing pages exactly as a real local user sees them, and how cloaking tries to hide from you.

Ad & Brand

Explore more