java—如何使用编程api向livy(在knox下)提交作业?

mlmc2os5  于 2021-05-17  发布在  Spark
关注(0)|答案(0)|浏览(311)

livy docs的代码运行良好:

LivyClient client = new LivyClientBuilder()
    .setURI(new URI("http://sample.ru:8999"))
    .build();
String piJar = "C:\\Users\\e.makrushin\\IdeaProjects\\untitled-maven\\out\\artifacts\\untitled_maven_jar\\untitled-maven.jar";
try {
    String date = "2020.01.07";
    System.err.printf("Job jar file %s uploading to the Spark context...\n", piJar);
    client.uploadJar(new File(piJar)).get();
    System.err.println("Job running at date " + date + " ...");
    double cheques = client.submit(new ChequesJob(date)).get();
    System.out.println("Job finished. " + cheques + " cheques found.");
} finally {
    client.stop(true);
}

devops通过knox路由了一个livy网关:

<service role="LIVYSERVER" name="livy" version="0.1.0">
  <routes>
    <route path="/livy/v1/sessions">
        <rewrite apply="LIVYSERVER/livy/addusername/inbound" to="request.body"/>
    </route>
    <route path="/livy/v1/**?**"/>
    <route path="/livy/v1"/>
    <route path="/livy/v1/"/>
  </routes>
  <dispatch classname="org.apache.knox.gateway.livy.LivyDispatch"/>
</service>

rewrite.xml:
<rules>
  <rule name="LIVYSERVER/livy/user-name">
    <rewrite template="{$username}"/>
  </rule>
  <rule dir="IN" name="LIVYSERVER/livy/root/inbound" pattern="*://*:*/**/livy/v1">
    <rewrite template="{$serviceUrl[LIVYSERVER]}"/>
  </rule>
  <rule dir="IN" name="LIVYSERVER/livy/path/inbound" pattern="*://*:*/**/livy/v1/{path=**}?{**}">
    <rewrite template="{$serviceUrl[LIVYSERVER]}/{path=**}?{**}"/>
  </rule>
  <filter name="LIVYSERVER/livy/addusername/inbound">
    <content type="*/json">
      <apply path="$.proxyUser" rule="LIVYSERVER/livy/user-name"/>
    </content>
  </filter>
</rules>rewrite template="{$serviceUrl[LIVYSERVER]}/"/>
    </rule>
    <rule dir="IN" name="LIVYSERVER/livy/inbound/path" pattern="*://*:*/**/livy/{**}">
        <rewrite template="{$serviceUrl[LIVYSERVER]}/{**}"/>
  </rule>
</rules>

livy会话列表现在可以在https://knox-gateway:8443/gateway/default/livy/v1/session
之后,正如预期的那样,此代码停止工作:

Exception in thread "main" java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at org.apache.livy.client.http.HttpClient.propagate(HttpClient.java:182)
    at org.apache.livy.client.http.HttpClient.<init>(HttpClient.java:82)
    at org.apache.livy.client.http.HttpClientFactory.createClient(HttpClientFactory.java:37)
    at org.apache.livy.LivyClientBuilder.build(LivyClientBuilder.java:130)
    at Main.main(Main.java:14)

我在livy文档中找不到任何关于使用编程api设置授权的内容。我应该向调用代码或配置文件中添加什么以使此代码再次工作?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题