Zaɓin wakili na zagaye a Colly tare da SetProxyFunc

Yi amfani da SetProxyFunc don zaɓin zagaye, tare da lura da tasirin keep-alive, maye gurbin transport da backend da aka kwafa.

collector := colly.NewCollector()
switcher, err := proxy.RoundRobinProxySwitcher(
    "http://proxy-a.example:8080",
    "http://proxy-b.example:8080",
)
if err != nil {
    return err
}
collector.SetProxyFunc(switcher)
err = collector.Visit("https://www.example.com/")

Yi amfani da github.com/gocolly/colly/v2 da fakitin proxy nasa a cikin aikin aikace-aikacen, tare da kiyaye maganin kuskure na yau da kullun. Saƙa mai zaɓin kafin buƙatu ko ziyarar async su fara. Jerin wakilan HTTP yana samar da ƴan takara na jama'a don shigarwar endpoint na mai sauyawa.

SetProxy yana zaɓar tsayayyen endpoint; SetProxyFunc yana barin transport na HTTP ya zaɓa ga kowace buƙata. Wannan yana guje wa maye gurbin haɗin kai da aka raba a cikin kowace OnRequest callback.

A cikin wannan aiwatarwa, SetProxyFunc kuma yana saita DisableKeepAlives zuwa true. Yana sabunta *http.Transport da ke akwai, amma yana maye gurbin wani nau'in transport daban da sabon HTTP transport. Ka adana duk wani hali na RoundTripper na musamman da gangan maimakon ɗauka cewa zai ci gaba.

Collector.Clone yana raba HTTP backend, don haka canza wakilin kwafi na iya canza wannan backend ɗin da aka raba. Yanke shawara kan mallaka kafin aiki na lokaci ɗaya. Zaɓin zagaye yana juya endpoint ɗin da aka bayar, ba fitattun shaidar fita da aka tabbatar ba ko samuwar da aka bincika da kai.

Labarai masu alaƙa