La wadaag barkada proxy ee u dhexeeya dunta Python iyadoo la isticmaalayo saf

Marka dunto badan ay wadaagaan barkad yar oo proxy, doorashada gelitaan random ah waxay siin kartaa laba hawlgalka firfircoon isla proxy. Safku wuxuu hayn karaa gelitaannada hadda diyaar u ah amaah:

from contextlib import contextmanager
from queue import Queue

available = Queue()
for endpoint in dict.fromkeys(proxy_urls):
    available.put(endpoint)

@contextmanager
def borrow_proxy():
    endpoint = available.get(timeout=5)
    try:
        yield endpoint
    finally:
        available.put(endpoint)

def process_url(url):
    with borrow_proxy() as endpoint:
        return fetch_one(url, endpoint)

Dhis safka hal mar, ka hor inta aadan bilaabin dunta shaqada. proxy_urls waxa uu ka kooban yahay URL-yada dhamaystiran ee barta dhammaadka. Musharrixiinta dadweynaha waxay ku jiraan liiska proxy ee bilaashka ah; isticmaal borotokoolka iyo qaabka URL ee fetch_one taageero. Shaqada fetch_one ee isku midka ah waa in ay dhammaysato hawlgalkeeda shabakada oo ay xirto jawaab kasta ka hor inta aysan soo noqon. Haddii kale amaahdu waxay dhammaan lahayd inta xiriirku weli isticmaalayo proxy-gaas.

Queue.get waxay si atomic ah uga saartaa hal gelitaan diyaar ah. Barkad la dhammeeyay waxay sugtaa halkii ay si isdaba joog ah u baari lahayd calaamadaha la wadaago. Haddii aanay gelitaan iman shan ilbiriqsi gudahood, queue.Empty waxay gaadhaa wicitaanka; dooro siyaasadda dib-u-dayactirka, guuldarada ama joojinta codsiga natiijadaas.

Block finally waxay soo celisaa gelitaan la qaatay ka dib guul ama ka reeb. Ma caddaynayso in barta dhammaadka ee guuldareysatay ay caafimaad qabto. Siyaasadaha karantiinka, beddelka iyo qaboojinta waxay u baahan yihiin waddo cad halkii si aamusan loo tuuri lahaa proxy kasta oo arka khalad ku meel gaar ah.

Ka saarista nuqullada isku midka ah ee xargaha URL waxay ka hortagtaa gelitaannada labanlaabka ah ee safka. URL-yo kala duwan ayaa weli tilmaami kara isla proxy jir ama bixitaan; si ula kac ah u kooxee haddii gaar ahaansho ay tahay in lagu dabaqo kheyraadkaas la wadaago. Safku wuxuu isku duwaa dunta habkan, ma aha shaqaalaha mishiinnada kale.