The Layers Behind a Proxied Mechanize HTTPS Request

See how Mechanize, LWP's HTTPS protocol module and the SSL backend cooperate to route, tunnel and verify one HTTPS request.

Set the route on the Mechanize object before fetching:

use WWW::Mechanize;

my $mech = WWW::Mechanize->new(timeout => 30);
$mech->proxy(
    ['http', 'https'],
    'http://proxy.example.net:8080',
);

Use the intended HTTP listener in the example; for permitted public-target requests, candidates are in the HTTP proxy list. The array names destination schemes. The http:// endpoint supplies an ordinary CONNECT route for HTTPS: HTTP to the proxy, tunnel to the target, then target TLS. If deployment uses environment settings instead, $mech->env_proxy is the alternative; importing them afterward can change a route assigned in code.

Mechanize inherits LWP::UserAgent networking rather than providing a separate HTTPS proxy engine:

  • Mechanize supplies links, forms and current-page behavior.
  • LWP::UserAgent and its HTTP transport select the route and arrange CONNECT.
  • LWP::Protocol::https supplies HTTPS handling and upgrades the connection.
  • Net::HTTPS with its SSL backend supplies the TLS-capable socket, normally IO::Socket::SSL in modern configurations.

An installed SSL backend does not itself select a proxy. Keep the original HTTPS hostname in the URL: CONNECT acceptance does not verify that site's certificate.

When maintaining old components, distinguish these boundaries. libwww-perl 6.06 and the separately distributed LWP::Protocol::https 6.06 introduced coordinated fixes for normal HTTP CONNECT. The UserAgent version alone does not identify the HTTPS module. LWP::Protocol::https 6.15 added a fix for TLS-enabled proxy endpoints, whose https:// scheme adds TLS to the proxy before target TLS.

Older Net::SSL/Crypt::SSLeay configurations can instead use that backend's documented HTTPS_PROXY environment mechanism. Do not apply its recipe to the modern IO::Socket::SSL path, where LWP owns routing.

Use the first failed stage to select the component: a missing HTTPS handler concerns the protocol module; rejected CONNECT concerns route, credentials or policy; a post-tunnel certificate error concerns destination identity and trust. Retain verification while changing the component supported by that evidence.

Allakkat attuumassuteqartut