Ku dheji dejinta proxy-ga Scrapy gudaha skript CrawlerProcess
U gudbi get_project_settings() CrawlerProcess marka bilowga madaxbanaan uu isticmaalo isla middleware-ka soo dejinta ee scrapy crawl.
Caan (spider) wuxuu isticmaali karaa proxy-ga la filayo hoosta scrapy crawl oo si toos ah ayuu u socdaa marka skript Python uu bilaabo. Hubi sida skriptkaas u dhiso dejintiisa ka hor intaadan beddelin server-ka proxy.
Loogu talagalay mashruuc Scrapy jira iyo caaradiisa diiwaangashan:
from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings
settings = get_project_settings()
process = CrawlerProcess(settings=settings)
process.crawl('catalog')
process.start()Ku beddel catalog magaca caanka mashruuca. Mashruuca waa in laga helaa iyada oo loo marayo scrapy.cfg, ama iyada oo loo marayo SCRAPY_SETTINGS_MODULE si sax ah loo dejiyay oo tilmaamaya module-ka dejinta ee la soo dejin karo.
Dhisidda kaliya CrawlerProcess() ma soo dejinayso faylka dejinta mashruuca sababtoo ah skriptku wuxuu sidoo kale soo dejiyaa fasalka caanka. Sidaa darteed middleware-ka gaarka ah ee lagu shiday DOWNLOADER_MIDDLEWARES ayaa laga yaabaa inuu ka maqnaado socodkaas.
Dejinta soo dejintu waxay bixisaa qaybaha la habeeyay; ma abuurto cinwaan proxy. Middleware-ka proxy HTTP ee Scrapy wali wuxuu u baahan yahay proxy-ga deegaanka caadiga ah ama qiime request.meta['proxy'] cad. Hubi in HTTPPROXY_ENABLED uusan damin. Baaritaanka bogagga dadweynaha ee la ogolaaday, liiska proxy-ga HTTP wuxuu bixiyaa musharraxiin qiimaha proxy HTTP ee lagu bixiyo metadataas ama deegaankaas.
Isbarbardhig dejinta la soo dejiyay iyo middleware-ka firfircoon ee bilowga iyo socodka khadka amarka. Haddii ay iswaafaqaan, sii wad metadata proxy-ga codsiga la doortay iyo khaladka rarista dhabta ah.