我提交了一个来自java的spark作业作为restful服务。我不断得到以下错误:
应用程序应用程序\u 1446816503326 \u 0098失败2次,原因是appattempt的am容器\u 1446816503326 \u 0098 \u000002已退出,exitcode:-1000有关更详细的输出,请检查应用程序跟踪page:http://ip-172-31-34-108.us-west-2.compute。internal:8088/proxy/application_1446816503326_0098/then,单击指向每次尝试日志的链接。诊断:java.io.filenotfoundexception:file file:/opt/apache tomcat-8.0.28/webapps/restfulexample/web-inf/lib/spark-yarn\u 2.10-1.3.0.jar不存在,尝试失败。应用程序失败。 spark-yarn_2.10-1.3.0.jar
文件在文件室里 lib
文件夹。
这是我的程序。
package SparkSubmitJava;
import org.apache.spark.deploy.yarn.Client;
import org.apache.spark.deploy.yarn.ClientArguments;
import org.apache.hadoop.conf.Configuration;
import org.apache.spark.SparkConf;
import java.io.IOException;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.core.Response;
@Path("/spark")
public class JavaRestService {
@GET
@Path("/{param}/{param2}/{param3}")
public Response getMsg(@PathParam("param") String bedroom,@PathParam("param2") String bathroom,@PathParam("param3")String area) throws IOException {
String[] args = new String[] {
"--name",
"JavaRestService",
"--driver-memory",
"1000M",
"--jar",
"/opt/apache-tomcat-8.0.28/webapps/scalatest-0.0.1-SNAPSHOT.jar",
"--class",
"ScalaTest.ScalaTest.ScalaTest",
"--arg",
bedroom,
"--arg",
bathroom,
"--arg",
area,
"--arg",
"yarn-cluster",
};
Configuration config = new Configuration();
System.setProperty("SPARK_YARN_MODE", "true");
SparkConf sparkConf = new SparkConf();
ClientArguments cArgs = new ClientArguments(args, sparkConf);
Client client = new Client(cArgs, config, sparkConf);
client.run();
return Response.status(200).entity(client).build();
}
}
任何帮助都将不胜感激。
暂无答案!
目前还没有任何答案,快来回答吧!