Proxy kasta oo codsi ah oo leh middleware-ka proxy HTTP ee Scrapy

Isticmaal metadata codsiga Scrapy iyo middleware-ka proxy-ga ka hor inta aanad koobin middleware-ka autentikaasha ee hore ee gaarka ah crawler-ka.

Middleware-ka proxy HTTP ee Scrapy wuxuu proxy ka akhriyaa metadata codsiga:

yield scrapy.Request(
    'https://example.com/',
    callback=self.parse,
    meta={'proxy': 'http://203.0.113.10:8080'},
)

Beddel barta ugu dambeysa ee dukumeentiga oo ka tag middleware-ka oo shaqeynaya. Liiska proxy HTTP wuxuu bixiyaa musharrixiinnada dadweynaha ee URL-ka codsigan. Tani waxay doorataa dariiqa codsiga; barkada proxy waxay u baahan tahay macquul doorasho gaar ah.

Autentikaashu waxay raacdaa barta ugu dambeysa ee la doortay

Middleware-ka ku dhex jira wuxuu maareeyaa aqoonsiga lagu bixiyo URL-ka proxy-ga la taageeray iyo madaxa oggolaanshaha la xiriira. Ka bax xuruufta URL-ka ee la kaydiyay oo ka fogow inaad URL-kaas diiwaangeliso.

Middleware gooni ah waa inuusan hayn madaxa Basic ee proxy-gii hore ka dib marka barta ugu dambeysa isbeddesho. Hal lakab ha maareeyo dariiqa iyo aqoonsigiisa wadaag.

Habka doorashada iyo caadiga

Haddii middleware-ka soo dejinta ee gaarka ah uu doorto proxy-ga, orod ka hor inta middleware-ka proxy-ga ku dhex jira uusan ka shaqeyn codsiga. Hubi nidaamka iyo API-ga nooca la rakibay; soo dejimaha scrapy.contrib ee duugoobay waxay ka tirsan yihiin noocyadii hore.

Deegaanka habsocodka wuxuu kaloo bixin karaa caadi gaar u ah qorshaha. Codsi si ula kac ah toos ah, isticmaal dhaqanka gaarka ah ee la diiwaangeliyay ee metadata proxy: fure maqan macnaheedu maaha mar walba gelitaan toos ah.

Baar natiijada dhabta ah. 407 waxay ku saabsan tahay autentikaasha proxy-ga, halka xaddidaadda xawaaraha ama diidmada gelitaanka websaydhku ay iman karaan ka dib marka dariiqu si sax ah u shaqeeyay.

Maqaallo la xiriira