Siiso resolvir-ka baakadaha ee spark-shell borotokoolkiisa xilliga kicirada

spark-shell --driver-java-options "-Dhttp.proxyHost=proxy.example -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy.example -Dhttps.proxyPort=8080" --packages group:artifact:version

Marka la kiciro spark-shell dhinaca macmiilka, xulashooyinkan JVM ee darawalka waxay yimaadaan ka hor xallinta ku-tiirsanaanta. Ku beddel barta ugu dambeysa ee tusaalaha iyo group:artifact:version qiimayaasha loogu talagalay adigoo isticmaalaya xulashooyin ay taageerto nooca Spark ee rakiban. Magaca martigeliyaha iyo dekedda Java waa sifooyin gooni ah. Soo dejinta ku-tiirsanaanta dadweynaha ee la oggolaaday, liiska borotokoolka HTTP wuxuu soo bandhigaa musharrixiinta beerarkaas.

Isticmaal meeshan marka --packages guuldareysto ka hor inta aan shell la helin. Koodhka codsiga ee dambe lama gelin karo ma geli karo soo dejinta hore. Spark wuxuu kala soocaa xulashooyinka kicinta iyo dejinta barnaamijka; qaabka macmiilka JVM darawalka ayaa horey u jiray marka koodhka codsiga uu socdo.

Qaabka geynta ayaa sidoo kale muhiim u ah spark.driver.extraJavaOptions: darawalka wuxuu ku shaqeyn karaa mashiin kale, kaas oo u baahan waddiisa u gaar ah borotokoolka. Codsiga API ee dambe ee executor waa geeddi-socod kale iyo goob shabakadeed kale; dejinta executors ma dejiso xallinta bilowga.

Marka xulashooyinka kicinta ay jiraan laakiin soo dejintu guuldareysto

Raac magaca martigeliyaha kaydka, nooca codsiga iyo qaladka ugu horreeya. Xogta iyo farshaxanka waxay ka iman karaan martigeliyo badan. URL kayd oo Maven la jaanqaadaya ma caddeeyo in Spark uu akhriyo settings.xml ee amarka Maven ee guuleysta isticmaalo.

Soo-dejiyaha Java waa inuu taageero habka borotokoolka iyo xaqiijinta loo baahan yahay. Sifooyinka martigeliyaha/dekedda ma bixiyaan qorshe kasta, iyo sifada http.proxyUsername ee la hindisay ma noqoto mid wax ku ool ah kaliya sababtoo ah Java wuu aqbalay magaca. Hayso wadaajinta horay iyo soo dejinta gaar ah ee guuldareystay markaad baarayso marxaladdan.