我正在尝试用livy创建spark交互式会话,并且我需要添加一个lib,比如我在hdfs中丢失的jar(请参阅我的代码)。但会话已停止,日志在下面。
代码:
client = LivyClient('http://sandbox.c4e.kyomei.fr:10500')
session = client.create_session(SessionKind.SPARK , jars = ['hdfs://sandbox-hdp.hortonworks.com:8020/tmp/tsa2-assembly-0.1.jar'] )
日志:
> 20/05/09 01:43:48 INFO LineBufferedStream: Exception in thread "main"
> scala.reflect.internal.FatalError: object Predef does not have a
> member classOf 20/05/09 01:43:48 WARN RSCClient:
> Error stopping RPC.
> io.netty.util.concurrent.BlockingOperationException:
> DefaultChannelPromise@786d2cd8(uncancellable)
>
> at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
>
> at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
>
> at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:230)
> ....
> Exception in thread "Thread-32" java.io.IOException: Stream closed
>
> at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)
>
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:283)
>
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> 20/05/09 01:43:48 WARN ContextLauncher: Child process exited with code
20/05/09 01:43:48 ERROR SparkProcApp: job was killed by user
暂无答案!
目前还没有任何答案,快来回答吧!