Two Downloads, Two Proxies, One curl Command

Use curl transfer boundaries to assign separate proxies and output files without an ambiguous command line.

Putting two URLs after one --proxy option sends both through the same proxy. When each download needs its own route, give curl a transfer boundary:

curl --proxy http://203.0.113.10:8080 \
  --output first.html https://example.com/ \
  --next \
  --proxy http://203.0.113.20:8080 \
  --output second.html https://example.org/

Both proxy addresses are placeholders; choose the HTTP endpoints from your service or the HTTP proxy list. The --next option starts another transfer with a fresh set of local options. State the second proxy explicitly. An option you supplied before the boundary may need to be supplied again afterward.

This is a sequential example. It makes the pairing of destination, output file, and proxy easy to inspect. curl also supports parallel transfers, but parallelism changes scheduling; it does not choose a proxy for each URL automatically.

Be careful with a text file of pairs

A line containing proxy-address destination-url is your application's format. curl does not infer those column meanings from arbitrary text. A shell loop, a generated curl configuration, or another small dispatcher must turn each record into explicit options.

Preserve arguments as arguments. Building a command by concatenating untrusted URLs can let spaces or shell characters change its meaning. For repeated jobs, curl's documented configuration-file syntax is easier to review than a long string assembled by hand.

Record the result of each transfer separately. A successful first download says nothing about the route or response of the second. Keep timeouts and failure handling attached to the transfer that needs them, and inspect the boundary when a setting appears to have disappeared.

Ũhoro ũhĩtĩte