Give a Twisted Proxy Request a Deadline for Its Body

A ProxyAgent can connect quickly and then wait indefinitely for a response body. Its proxy endpoint’s connection timeout does not describe that later wait.

Set a connection bound on the endpoint when building the agent. For the public HTTP request below, an unauthenticated candidate from the HTTP proxy list can replace the address if public intermediaries are permitted:

endpoint = TCP4ClientEndpoint(reactor, 'proxy.example', 8080,
                              timeout=5)
agent = ProxyAgent(endpoint, reactor=reactor)

For a small response you intend to collect in memory, include body reading in the Deferred chain before adding its deadline:

from twisted.web.client import readBody

d = agent.request(b'GET', b'http://example.com/')
d.addCallback(readBody)
d.addTimeout(10, reactor)
d.addCallback(lambda body: print(len(body)))
d.addErrback(lambda failure: print(failure.getErrorMessage()))

The fragment assumes the application already has its reactor and configured agent. ProxyAgent’s request Deferred yields a response object; readBody returns the Deferred for the body. Returning it from the callback makes the chain wait for that operation too.

Placement matters. Twisted applies addTimeout to the work represented by callbacks installed before it. Adding a body reader afterward would describe a different boundary. In this arrangement the timer starts when addTimeout is called and covers the pending request and body work in that chain.

Use a streaming consumer for large or unbounded bodies and represent its completion in the same way. A client-side deadline limits how long your application waits; it does not reconfigure the proxy’s own upstream connection timeout.