Wareejiyeyaasha proxy ee httr ee R iyo falanqaynta HTML-ka la soo dejistay

Beddel safka proxy ee codsi kasta oo httr ah oo u gudbi baytyada la soo dejistay xml2 iyo rvest.

Hayso doorashada proxy-ga hawsha soo dejinta. Marka httr soo celiyo bogga, falanqee baytyada jawaabtaas halkii aad URL-ka u gudbin lahayd soo-dejiye kale.

Iyadoo la bixiyay vector urls iyo data frame proxies aan madhnayn oo leh tiirarka host iyo port tiro dhan:

pages <- vector("list", length(urls))
stopifnot(nrow(proxies) > 0L)

for (i in seq_along(urls)) {
  row <- ((i - 1L) %% nrow(proxies)) + 1L
  response <- httr::GET(
    urls[[i]],
    httr::use_proxy(proxies$host[[row]], proxies$port[[row]]),
    httr::timeout(30)
  )
  httr::stop_for_status(response)
  pages[[i]] <- xml2::read_html(httr::content(response, as = "raw"))
}

Codsiyada bogagga dadweynaha ee lagu oggolaaday dhexdhexaadiyeyaasha dadweynaha, liiska proxy-ga HTTP wuxuu bixiyaa musharraxiin tiirarka host iyo port. Tilmaamuhu wuxuu ka wareegaa safka koowaad ee proxy ilaa kan ugu dambeeya kadibna wuu ku noqdaa bilowga. Wicitaan kastaa wuxuu leeyahay qaabeynta proxy u gaar ah; geeddi-socodka R wuu sii socon karaa.

Bogagga soo baxa waxaa ku jira dukumeenti HTML oo la falanqeeyay. Ku dabaq rvest::html_elements iyo hawlaha soo saarista loo baahan yahay dukumeentiyadaas. In URL la siiyo read_html waa inuu bilaabi lahaa soo-jiid cusub oo leh qaabeynta shabakadeed ee jidkaas.

Proxy-yada la xaqiijiyay, u gudbi akoonka la doortay iyada oo loo marayo doodaha username iyo password ee use_proxy. Barta kale ee dhamaadka ma dammaanad qaaddo IP bixid dadweyne kale, marka dooro barkada iyadoo loo eegayo dhaqanka bixidda dhabta ah ee adeegga.

Maqaallo la xiriira