Stop a Proxy Rotation Loop from Running Forever

A proxy pool can turn a simple outage into a busy loop. Pick an address, fail, pick another, fail again. If every endpoint is unreachable because your own network is down, changing addresses never fixes the cause.

Give the operation a budget before choosing the first proxy from your pool or the free proxy list: a finite number of attempts, bounded connection and read waits, and a deadline for the overall task. A Requests timeout controls its documented connection or read behavior; it does not by itself cap a loop that keeps issuing new requests.

Decide what earns another attempt

• A refused proxy connection can justify trying a different endpoint.
• A destination response needs its own interpretation. Receiving an HTTP error proves that some request path completed.
• A proxy-authentication failure calls for checking the endpoint and account rather than hammering the same credentials through a growing list.

Keep the original exception or status with the attempt record. A final message containing only “all proxies failed” hides whether the failures even had the same cause.

Health-checking each proxy before every real request adds another request and still cannot guarantee that the next one will succeed. Reuse recent health information where it fits the task, then let the real operation report its own result.

A retry can repeat the work

For a read-only fetch, moving to another proxy is usually simpler than replaying a submission. If a connection drops after sending a purchase or form submission, the destination might already have processed it. Follow the service's idempotency mechanism and the HTTP retry rules before replaying work.

Once the budget is exhausted, return a useful failure. A job that stops with evidence is easier to repair than one that quietly keeps rotating until someone notices.