Daadin diiwaannada CSV ee dhowr sadar leh iyadoo la adeegsanayo Python Requests iyo csv.reader

U daadi jawaabta proxy-ga gudaha csv.reader oo ilaali xadka diiwaannada halkii aad kala qaybin lahayd sadar kasta oo jireed delimiter-ka.

stream=True waxay u ogolaataa habaynta inay bilaabato ka hor inta aan jirka jawaabta proxy-ga si buuxda loo soo dejisan. CSV ahaan, u gudbi isku-celiye sadar qoraal ah csv.reader si delimiter-yada ku jira xardho iyo diiwaannada dhowr sadar leh ay u ahaadaan xog CSV.

Fayl UTF-8 tusaale ah oo isticmaalaya qoraal-xariir:

import csv
import requests

routes = {'https': 'http://proxy.example:8080'}
with requests.get(
    'https://data.example/observations.csv',
    proxies=routes,
    stream=True,
    timeout=(5, 30),
) as response:
    response.raise_for_status()
    response.encoding = 'utf-8'
    lines = (
        line + '\n'
        for line in response.iter_lines(
            chunk_size=65536, decode_unicode=True)
    )
    for record in csv.reader(lines, delimiter=';'):
        if record and record[0] == 'wanted':
            print(record)
            break

Dooro codeynta iyo lahjadda dhabta ah ee faylka. Soo-dejis dadweyne CSV oo ku socda waddo proxy dadweyne oo la ogolaaday, liiska proxy HTTP wuxuu bixiyaa musharrixiinta URL-ka routes. Nuqtan waxay caadi ka dhigtaa dhammaadka sadarrada jireed LF: iter_lines wuu ka tiraa, jeneraatorna wuxuu ku darayaa sadar cusub falanqeeyaha CSV. Taasi waxay muhiim tahay marka goob ku jirta xardho ay ku sii socoto sadar jireed kale.

Hawlaha u baahan in si byte-ahaan sax ah loo ilaaliyo xarfaha CRLF ee la geliyey, isticmaal hab-dhaqan duudduubis qoraal oo daadin ah oo ilaaliya dhammaadka sadarradaas. Caadi-ka-dhigista ula kac ah ee tusaalaha waxay ku habboon tahay habaynta diiwaan-ku-salaysan halkaas oo LF uu yahay matalaadda loogu talagalay.

Hal jawaab, akhriso badan

Isku-celiyaha wuxuu cunaa qaybo ka mid ah hal jawaab HTTP. Ma soo saaro codsi cusub sadar kasta. Codsiyada HTTP Range waa hab kale mana aqoonsadaan si dabiici ah xadka diiwaannada CSV.

Wicitaanka response.text ama ururinta dhammaan safafka liis ayaa dib ugu soo celin lahaa nuxurka oo dhan xusuusta. Daadintu weli waxay u baahan tahay meel bufferyada gaadiidka iyo diiwaanka hadda, kaas oo isagu yeelan kara goob weyn.

Macnaha jawaabtu wuxuu xiraa jirka marka wareeggu baxo, oo ay ku jirto isku-dhac hore. Waqtiga akhrisku wuxuu xaddidaa sugitaanka xogta soo gasha, ma aha cimriga guud ee wareejinta fayl weyn.

Maqaallo la xiriira