Ukusakaza amarekhodi e-CSV anemigqa eminingi nge-Python Requests ne-csv.reader

Sakaza impendulo nge-proxy ku-csv.reader futhi ugcine imingcele yamarekhodi esikhundleni sokuhlukanisa umugqa ngamunye womzimba ku-delimiter.

I-stream=True ivumela ukucubungula ukuqala ngaphambi kokuba umzimba wempendulo nge-proxy ulandwe ngokuphelele. Nge-CSV, dlulisa isiphindaphindi semigqa yombhalo ku-csv.reader ukuze izihlukanisi ezingaphakathi kwezicaphuno namarekhodi anemigqa eminingi kuhlale kuyidatha ye-CSV.

Ngefayela le-UTF-8 eliyisibonelo elisebenzisa amaphuzu-nokukhefana:

import csv
import requests

routes = {'https': 'http://proxy.example:8080'}
with requests.get(
    'https://data.example/observations.csv',
    proxies=routes,
    stream=True,
    timeout=(5, 30),
) as response:
    response.raise_for_status()
    response.encoding = 'utf-8'
    lines = (
        line + '\n'
        for line in response.iter_lines(
            chunk_size=65536, decode_unicode=True)
    )
    for record in csv.reader(lines, delimiter=';'):
        if record and record[0] == 'wanted':
            print(record)
            break

Khetha ukufakwa kwekhodi nolimi lwesifunda lwangempela lwefayela. Yokulanda i-CSV yomphakathi emzileni we-proxy yomphakathi evunyelwe, uhlu lwe-proxy ye-HTTP lunikeza abazongenela i-URL ye-routes. Le nguqulo ijwayeza iziphetho zemigqa yomzimba zibe yi-LF: i-iter_lines iyazisusa, futhi umakhi wengeza umugqa omusha futhi womhlaziyi we-CSV. Lokho kubalulekile lapho inkambu engaphakathi kwezicaphuno iqhubeka emugqeni womzimba ohlukile.

Emisebenzini edinga ukugcinwa okunembile ngamabhayithi kwezinhlamvu ze-CRLF ezifakiwe, sebenzisa uhlelo lokusonga umbhalo olusakazayo olugcina lezo ziphetho zemigqa. Ukujwayezwa ngamabomu kwesibonelo kufanele ukucubungula okugxile kumarekhodi lapho i-LF iwukumelela okuhlosiwe.

Impendulo eyodwa, ukufunda okuningi

Isiphindaphindi sidla izingcezu ezivela empendulweni eyodwa ye-HTTP. Asikhiphi isicelo esisha somugqa ngamunye. Izicelo ze-HTTP Range ziyindlela ehlukile futhi azikhonji imingcele yamarekhodi e-CSV ngokwemvelo.

Ukushaya i-response.text noma ukuqoqa yonke imigqa ohlwini kuzobuyisela konke okuqukethwe emqondweni. Ukusakaza kusadinga indawo yamabhafa okuthutha kanye nerekhodi lamanje, elingaqukatha inkambu enkulu ngokwalo.

Umongo wempendulo uvala umzimba lapho iluphu iphuma, kuhlanganise nokufanisa kusenesikhathi. Isikhathi sokufunda sikhawulela ukulinda idatha engenayo, hhayi impilo iyonke yokudluliswa kwefayile elikhulu.

Izihloko ezihlobene