Colly Round-Robin Proxy Selection with SetProxyFunc

Use SetProxyFunc for round-robin selection, with its keep-alive, transport replacement and cloned-backend effects in view.

collector := colly.NewCollector()
switcher, err := proxy.RoundRobinProxySwitcher(
    "http://proxy-a.example:8080",
    "http://proxy-b.example:8080",
)
if err != nil {
    return err
}
collector.SetProxyFunc(switcher)
err = collector.Visit("https://www.example.com/")

Use github.com/gocolly/colly/v2 and its proxy package in the application's function, retaining normal error handling. Configure the selector before requests or asynchronous visits start. The HTTP proxy list supplies public candidates for the switcher’s endpoint inputs.

SetProxy chooses a fixed endpoint; SetProxyFunc lets the HTTP transport choose per request. It avoids replacing shared configuration inside each OnRequest callback.

In this implementation, SetProxyFunc also sets DisableKeepAlives true. It updates an existing *http.Transport, but replaces a different transport type with a new HTTP transport. Preserve any custom RoundTripper behavior deliberately rather than assuming it survives.

Collector.Clone shares the HTTP backend, so changing a clone's proxy can change that shared backend. Settle ownership before concurrent work. Round-robin selection rotates supplied endpoints, not verified public exit identities or independently checked availability.

Erthyglow kevrennek