问题-无法使用Spark Cassandra连接器1.5.0连接Cassandra 3.0
背景-我试图通过使用提供的Spark cassandra连接器1.5.0从Spark 1.5.0连接Cassandra 3.0,但我得到以下错误-
根据DataStax Spark Cassandra连接器document,它表示Spark连接器1.5可以从Spark 1.5.0/1.6.0用于Cassandra 3.0。
你能告诉我我是不是漏掉了什么步骤吗?
尝试过的方法
1.我尝试从Spark流和核心jar中排除Guava依赖项
1.在“pom.xml”中添加了单独的Guava依赖项
先谢谢你。
16/04/26 09:45:07 WARN TaskSetManager: Lost task 4.0 in stage 1.0 (TID 16, ip-172-31-23-23.ec2.internal): java.lang.ExceptionInInitializerError
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.clusterBuilder(CassandraConnectionFactory.scala:35)
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createCluster(CassandraConnectionFactory.scala:87)
at
---
--
Caused by: java.lang.IllegalStateException: Detected Guava issue #1635 which indicates that a version of Guava less than 16.01 is in use. This introduces codec resolution issues and potentially other incompatibility issues in the driver. Please upgrade to Guava 16.01 or later.
at com.datastax.driver.core.SanityChecks.checkGuava(SanityChecks.java:62)
at com.datastax.driver.core.SanityChecks.check(SanityChecks.java:36)
at com.datastax.driver.core.Cluster.<clinit>(Cluster.java:67)
... 23 more16/04/26 09:45:07 WARN TaskSetManager: Lost task 4.0 in stage 1.0 (TID 16, ip-172-31-23-23.ec2.internal): java.lang.ExceptionInInitializerError
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.clusterBuilder(CassandraConnectionFactory.scala:35)
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createCluster(CassandraConnectionFactory.scala:87)
at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:153)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:148)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:148)
at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31)
1条答案
按热度按时间7uzetpgm1#
连接器附带的Guava版本和spark附带的版本之间存在已知的冲突。您是如何尝试对Guava库进行着色的?
尝试将以下代码添加到您的build.sbt文件中: