Yabelana ngephuli ye-proxy phakathi kwezintambo ze-Python ngomugqa

Lapho izintambo eziningi zabelana ngephuli encane ye-proxy, ukukhetha ingeno engahleliwe kungase kunikeze imisebenzi emibili esebenzayo i-proxy efanayo. Umugqa ungagcina izingeno ezikhona okwamanje zokuboleka:

from contextlib import contextmanager
from queue import Queue

available = Queue()
for endpoint in dict.fromkeys(proxy_urls):
    available.put(endpoint)

@contextmanager
def borrow_proxy():
    endpoint = available.get(timeout=5)
    try:
        yield endpoint
    finally:
        available.put(endpoint)

def process_url(url):
    with borrow_proxy() as endpoint:
        return fetch_one(url, endpoint)

Yakha umugqa kanye, ngaphambi kokuqala izintambo zokusebenza. i-proxy_urls iqukethe ama-URL aphelele wendawo yokugcina. Abantu abavela emphakathini batholakala ohleni lwama-proxy amahhala; sebenzisa iphrothokholi kanye nefomethi ye-URL esekelwa yi-fetch_one. Umsebenzi we-fetch_one ohambisanayo okhona kumele uqede umsebenzi wawo wenethiwekhi futhi uvale noma yimuphi umpendulo ngaphambi kokubuya. Uma kungenjalo isivumelwano sokuqashisa singaphela ngesikhathi ukuxhumana kusasebenzisa leyo proxy.

I-Queue.get isusa ingeno eyodwa ekhona ngendlela ye-athomikhi. Iphuli eliphelile lilinda esikhundleni sokuhlola amafulegi abiwekanayo kaningi. Uma kungekho ngeno efikayo phakathi kwemizuzwana emihlanu, i-queue.Empty ifinyelela kumfoni; khetha inqubomgomo yohlelo yokuphinda uzame, yokwehluleka noma yokukhansela yalowo mphumela.

Ibhulokhi ye-finally ibuyisela ingeno etholiwe ngemuva kokuphumelela noma kwe-exception. Ayimemezeli ukuthi indawo yokugcina ehlulekile inempilo. Izinqubomgomo ze-quarantine, zokushintsha kanye nokupholisa zidinga indlela ecacile esikhundleni sokulahla buthule yonke i-proxy ebona iphutha lesikhashana.

Ukususa ukuphindaphinda kwemigqa ye-URL efanayo kuvimbela izingeno eziphindwe kabili kumugqa. Ama-URL ahlukene asengakhomba ku-proxy efanayo yomzimba noma ekuphumeni; bawahlanganise ngamabomu uma ukukhethekile kufanele kusebenze kuleyo nsiza yabiwe. Umugqa uxhumanisa izintambo kule nqubo, hhayi izisebenzi kweminye imishini.