Handle Failed Proxy Requests with asyncio.gather

asyncio.gather with return_exceptions=True puts task exceptions into its result list. Awaiting the batch without an exception therefore does not mean every proxy request succeeded.

Keep the destination and chosen proxy alongside the submitted awaitables, then inspect every outcome:

outcomes = await asyncio.gather(*jobs, return_exceptions=True)

for (url, proxy), outcome in zip(routes, outcomes):
    if isinstance(outcome, BaseException):
        record_failure(url, proxy, outcome)
    else:
        record_success(url, proxy, outcome)

jobs contains the application’s request coroutines; routes contains their corresponding (url, proxy) pairs in the same order. The recording functions are the application’s own result handling. gather preserves input order even when completion order differs.

Classify a returned child CancelledError as cancellation rather than blaming its proxy. Cancellation of the outer gather still propagates to unfinished children; return_exceptions does not turn that cancellation into ordinary success.

HTTP status handling also belongs inside the request operation. aiohttp does not raise for every HTTP error by default, so return the status or apply raise_for_status according to the job’s policy.

Once outcomes are visible, the application can distinguish connection failures from destination responses and decide which work merits another attempt.