Xhuma i-proxy nezingcingo zezithombe ze-Scrapy

Hlobisa ama-Requests akhiqizwa yi-ImagesPipeline ukuze ukulandwa kwezithombe kuthole i-proxy ngaphandle kokwakha kabusha umqondo wezingcingo we-pipeline.

from scrapy.pipelines.images import ImagesPipeline

class ProxiedImagesPipeline(ImagesPipeline):
    def get_media_requests(self, item, info):
        for request in super().get_media_requests(item, info):
            request.meta['proxy'] = 'http://proxy.example:8080'
            yield request

Ku-Scrapy 2.13.3, shintsha ingeno ejwayelekile ye-ImagesPipeline ku-ITEM_PIPELINES ngendlela yokungenisa yale klasi encane. Gcina IMAGES_STORE nezinye izilungiselelo.

Ukushayela umzali kugcina inkambu ye-URL yesithombe kanye nama-callback alungiselelwe. Iklasi encane yengeza kuphela umzila wesicelo, osebenza nge-HttpProxyMiddleware enikwe amandla. Yonke i-URL yesithombe ihlala iyindawo yayo. Izicelo ezintsha zezithombe zomphakathi ezingenazo izifakazelo ze-proxy zingasebenzisa ingeno evela kuhlu lwe-proxy ye-HTTP lapho lowo mzila uvunyelwe.

Izicelo zamakhasi zigcina inqubomgomo yazo. Ngendawo ye-proxy yomhlaba wonke, meta={'proxy': None} yenza isicelo sekhasi esikhethiwe sibe nqo kule ndatshana.

Hlola ukuthi ukulanda kukhishwe ngempela. Ukushaywa kwe-cache ye-pipeline kungagwema isicelo senethiwekhi. Kokudluliswa okusha, qhathanisa amangeno emiphumela yezithombe eziqediwe nama-URL aceliwe: imisebenzi ephumelelayo ivela lapho, kuyilapho ukwehluleka kungashiya inkambu ye-URL yesithombe yokuqala igcwele. Landelela imiphumo engekho ngamalogi omlandi kanye ne-proxy ekhethiwe.

Izihloko ezihlobene