Clean an IP:Port Proxy List with Python

Put one IPv4 IP:port entry per line in proxies.txt. All entries must belong to one known proxy protocol. If you collect entries from the free proxy list, separate protocols before building this plain-text input. This cleanup accepts neither credential URLs nor CSV columns or IPv6 addresses.

Save this standard-library script as clean-proxies.py in the same directory:

from ipaddress import IPv4Address
from pathlib import Path

seen = set()
cleaned = []

for number, raw in enumerate(
    Path("proxies.txt").read_text(encoding="utf-8").splitlines(),
    start=1,
):
    text = raw.strip()
    if not text:
        continue
    try:
        host, port_text = text.rsplit(":", 1)
        host = str(IPv4Address(host))
        port = int(port_text)
        if not 1 <= port <= 65535:
            raise ValueError("port out of range")
    except ValueError:
        print(f"Skipping line {number}: invalid IPv4 endpoint")
        continue

    endpoint = (host, port)
    if endpoint not in seen:
        seen.add(endpoint)
        cleaned.append(f"{host}:{port}")

output = "\n".join(cleaned)
Path("proxies-clean.txt").write_text(
    output + ("\n" if output else ""),
    encoding="utf-8",
)
print(f"Saved {len(cleaned)} endpoints")

Run it from that directory:

uv run --no-project python clean-proxies.py

Inspect the skipped-line notices and proxies-clean.txt; the input file remains intact. The output preserves first-seen order and deduplicates on address plus numeric port. Two copies of 203.0.113.10:8080 and one 203.0.113.10:3128 produce two endpoints. Decimal port 08080 normalizes to 8080.

The documentation addresses illustrate formatting only. Valid IPv4 syntax includes private and special-purpose addresses, so a clean entry is not necessarily publicly reachable. Add an explicit address policy if you need a public-only list.

For mixed HTTP/SOCKS input, retain protocol information in separate files or a column; do not infer it from port 1080. Use a parser for bracketed IPv6 when that is the actual format. This script rejects unsupported input rather than guessing, and its output says nothing about endpoint availability, anonymity, country or speed. Those require controlled connection tests.