Scrapy wuxuu muujin karaa isu-wareejinta ka hor inta aanu raacin

Si ku meel gaar ah u muuji xaaladda isu-wareejinta bilowga ah iyo Location ee GET proxy ah iyada oo loo marayo metadata codsiga, iyadoo maaraynta isu-wareejinta laga soocay shaandhaynta khaladaadka HTTP.

GET proxy ah wuxuu gaari karaa server ka dibna raaci karaa isu-wareejinno ilaa guuldarada ugu dambeysa ay qariso jawaabta ugu horreysa. U muuji jawaabtaas bilowga ah si loo baaro:

request = scrapy.Request(
    'https://service.example/page',
    callback=inspect_redirect,
    meta={
        'proxy': 'http://proxy.example:8080',
        'dont_redirect': True,
        'handle_httpstatus_list': [301, 302, 303, 307, 308],
    },
)

Jadwal codsigan iyada oo loo marayo socodka bilowga ama callback ee caadiga ah ee caarada. Baaritaan bogga dadweynaha oo la oggolaaday awgiis, liiska proxy-yada HTTP wuxuu bixiyaa musharraxiin loogu talagalay URL-ka proxy ee codsiga. inspect_redirect waa inuu diiwaangeliyaa xaaladda jawaabta iyo madaxa Location halkii uu mar kale indho la'aan u raaci lahaa.

dont_redirect wuxuu joojiyaa isu-wareejinta otomaatigga ah ee codsigan. handle_httpstatus_list wuxuu u oggolaanayaa xaaladaha la magacaabay inay gaaraan callback halkii ay shaandhaynta khaladaadka HTTP diidi lahayd. Dejintaani waxay khuseeyaan mas'uuliyado kala duwan oo middleware.

Hubi waxa isu-wareejintu dooratay: URL canonical ah oo loogu talagalay, scheme ama magac martigeliye kale, bog gelitaan, ama wareeg soo noqnoqda. 302 la helay ka dib gaadiid guul leh waa u kuurgal ka duwan guuldarada furitaanka xiriirka proxy.

Marka maaraynta caadiga ah ee isu-wareejinta la soo celiyo, Scrapy wuxuu muujiyaa redirect_urls iyo redirect_reasons ee metadata codsiga jawaabta. Kuwani waxay ilaalin karaan wadada u horseedda natiijada ugu dambeysa iyagoon isu-wareejinno ka tegin la joojiyay crawling-ka oo dhan.

Jawaabta ugu dambeysa si otomaatig ah ma aha bogga ay codsiga u jeedday. Hayso meesha loo socdo iyo nuxurka jawaabta oo la socda baaritaanka dariiqa.

Maqaallo la xiriira