Python urllib: Combine Proxy Routing with Website Basic Authentication

Combine urllib routing and destination Basic authentication in one explicit opener without replacing the global opener twice.

Calling install_opener() twice does not add the second opener to the first. It replaces the module's default opener. If one object knows the proxy and another knows the website login, build a single opener containing the required handlers.

import os
from urllib.request import (ProxyHandler, HTTPBasicAuthHandler,
                            HTTPPasswordMgrWithDefaultRealm, build_opener)

passwords = HTTPPasswordMgrWithDefaultRealm()
passwords.add_password(None, 'https://example.com/private/',
                       os.environ['SITE_USER'], os.environ['SITE_PASSWORD'])
opener = build_opener(
    ProxyHandler({'https': 'http://203.0.113.10:8080'}),
    HTTPBasicAuthHandler(passwords),
)
with opener.open('https://example.com/private/report', timeout=10) as response:
    body = response.read()

The proxy and destination are examples. For a public service that may be reached through a public HTTP proxy, the HTTP proxy list provides candidate endpoints for the routing handler. This opener routes the HTTPS request through the HTTP proxy and handles the destination's Basic challenge. It does not configure proxy authentication.

If the proxy also requires a login, choose a supported proxy-authentication arrangement separately. A 407 at CONNECT time occurs before the destination can issue its own challenge. Adding website credentials to the password manager cannot satisfy that earlier exchange.

Calling opener.open() directly keeps the configuration with the operation. There is no need to change the module-wide default unless the application deliberately wants all convenience calls to share this opener.

Ũhoro ũhĩtĩte