CY
Scraping Frameworks

Colly (Go) proxy integration

Colly is the popular Go scraping framework and supports proxy switching via colly/proxy. Point its proxy switcher at your Aethyn endpoint to route collector requests through residential IPs.

Setup steps

  1. 1Install Colly: go get github.com/gocolly/colly/v2.
  2. 2Create a RoundRobinProxySwitcher with the Aethyn URL.
  3. 3Attach it with c.SetProxyFunc.
  4. 4Add targeting suffixes in the username.

Code

Colly with proxy switcher
package main

import (
    "log"
    "github.com/gocolly/colly/v2"
    "github.com/gocolly/colly/v2/proxy"
)

func main() {
    c := colly.NewCollector()
    rp, err := proxy.RoundRobinProxySwitcher(
        "http://aethyn-XXXXX-country-us:PASSWORD@proxy.aethyn.io:2099")
    if err != nil {
        log.Fatal(err)
    }
    c.SetProxyFunc(rp)

    c.OnHTML("title", func(e *colly.HTMLElement) {
        log.Println(e.Text)
    })
    c.Visit("https://example.com")
}

Replace aethyn-XXXXX and PASSWORD with your dashboard credentials. See the full targeting reference and protocol/port list.

Tips

  • Pass multiple proxy URLs to the switcher to rotate across configs.
  • Set c.Limit to control concurrency and delay for residential routes.
  • Aethyn rotates exits per request even with one proxy URL.

Colly (Go) proxy FAQ

Can Colly rotate multiple proxies?
Yes. RoundRobinProxySwitcher accepts several proxy URLs and cycles through them per request.

Ready to wire up Colly (Go)?

Get your credentials and route your first Colly (Go) request through residential IPs in minutes.

Docs, solutions & blog

Related integrations

All integrations