Extract Proxy Endpoints One Table Row at a Time

Parse each proxy row as a unit. Flattening every matching td across a page and grouping the results by a fixed count can pair an address from one row with a port from the next when a cell is missing or another table appears.

For HTML already obtained by the application, a table whose first two columns are IP address and port can be parsed as follows:

import ipaddress
from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
table = soup.select_one(table_selector)
if table is None:
    raise ValueError("Expected proxy table was not found")

endpoints = set()
for row in table.select("tbody tr"):
    cells = row.find_all("td", recursive=False)
    if len(cells) < 2:
        continue
    try:
        address = ipaddress.ip_address(cells[0].get_text(strip=True))
        port = int(cells[1].get_text(strip=True))
    except ValueError:
        continue
    if 1 <= port <= 65535:
        endpoints.add((str(address), port))

html and table_selector are application inputs. Choose the selector and column mapping from the source’s actual markup. The free proxy list is one source of endpoint records, but its rows should be inspected before choosing that mapping. Neither a historical table ID nor the claim that a page contains exactly 300 records is a lasting schema.

The set removes duplicate address-and-port pairs while preserving each pair’s association. An unexpected zero-result import should still be investigated: the table may have changed or the fetched document may be a challenge page rather than the listing.

Keep any advertised protocol and check-time fields alongside the endpoint when they matter. Extracting a plausible IP and port establishes formatting, not that the endpoint currently provides a working HTTP or SOCKS service.

Header or filter rows are not endpoints. A row containing only th cells is skipped, while empty or non-address fields fail validation. Do not discard the first and last row by position: either can be a real endpoint after the page layout changes.