我已编写shell脚本来运行spark作业:
/home/ajit/deviceAnalysis.sh
/usr/spark/bin/spark-submit --master yarn --deploy-mode cluster --packages org.mongodb.spark:mongo-spark-connector_2.11:2.4.1 --conf spark.yarn.maxAppAttempts=4 --conf spark.yarn.am.attemptFailuresValidityInterval=1h --class com.mmi.probes.analysis.app.StartApp --driver-memory 2g --queue deviceAnalysis ./DailyDeviceProbesAnalysis.jar
到目前为止一切正常( sh /home/ajit/deviceAnalysis.sh
). 但我想使用crontab每5分钟运行一次,它正在运行,但引发异常:
java.io.FileNotFoundException: File file:/home/ajit/.sparkStaging/application_1601983398886_0126/__spark_libs__9052532821082604838.zip does not exist
在crontab中 */10 * * * * cd /home/ajit && sh deviceAnalysis.sh
请帮帮我现在该怎么办
暂无答案!
目前还没有任何答案,快来回答吧!