cassandra 写入Amazon密钥空间时出现CodecNotFoundException

yzuktlbb  于 2022-11-05  发布在  Cassandra
关注(0)|答案(2)|浏览(200)

我正在尝试将Spark DF写入AWS Keyspaces。随机地,一些记录正在更新,一些记录抛出此异常

com.datastax.oss.driver.api.core.type.codec.CodecNotFoundException: Codec not found for requested operation: [INT <-> java.lang.String]
at com.datastax.oss.driver.internal.core.type.codec.registry.CachingCodecRegistry.createCodec(CachingCodecRegistry.java:609)
at com.datastax.oss.driver.internal.core.type.codec.registry.DefaultCodecRegistry$1.load(DefaultCodecRegistry.java:95)
at com.datastax.oss.driver.internal.core.type.codec.registry.DefaultCodecRegistry$1.load(DefaultCodecRegistry.java:92)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2276)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache.get(LocalCache.java:3951)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache.getOrLoad(LocalCache.java:3973)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4957)
at com.datastax.oss.driver.shaded.guava.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4963)
at com.datastax.oss.driver.internal.core.type.codec.registry.DefaultCodecRegistry.getCachedCodec(DefaultCodecRegistry.java:117)
at com.datastax.oss.driver.internal.core.type.codec.registry.CachingCodecRegistry.codecFor(CachingCodecRegistry.java:258)
at com.datastax.oss.driver.internal.core.data.ValuesHelper.encodePreparedValues(ValuesHelper.java:112)
at com.datastax.oss.driver.internal.core.cql.DefaultPreparedStatement.bind(DefaultPreparedStatement.java:158)

我的密钥空间表模式为

CREATE TABLE test_ks.test_table_ttl (
    consumer_id TEXT PRIMARY KEY,
    ttl_col map<text, frozen<tuple<text, text>>>
);

引发错误的代码块如下。

val rowKey =   // some string
val mapKey =   // some string
val mapValue = mapValueTupleType.newValue(tuple_value)
val mapData = ImmutableMap.builder().put(mapKey, mapValue).build()
batch.addStatement(prep_statement.bind(mapData, rowKey)) // <--- error on this line
3lxsmp7m

3lxsmp7m1#

尝试转换为RDD,然后写入。尝试将数据存储为json blob,而不是嵌套集合。

val myRdd = myDataframe.rdd

implicit val c = connectToKeyspaces

myRdd.saveToCassandra("aws_sample_keyspace","events_tables")
rmbxnbpk

rmbxnbpk2#

目前,AWS Keyspaces不允许冻结类型。这是Keyspaces中的一个错误,允许使用冻结类型创建表,但在插入过程中会抛出异常。
唯一可能的方法是使用@MikeJPR建议的JSON存储数据。

相关问题