我已经阅读了spark文档,但不确定如何使用spark cassandra连接器将javabean保存到表中?
public class NewImageMetadataRow implements Serializable {
private final String merchant;
private final String productId;
private final String url;
private final int width;
private final int height;
public NewImageMetadataRow(NewImageMetadataRow row) {
this.merchant = row.getMerchant();
this.productId = row.getProductId();
this.url = row.getUrl();
this.width = row.getWidth();
this.height = row.getHeight();
}
public String getMerchant() {
return merchant;
}
public String getProductId() {
return productId;
}
public String getUrl() {
return url;
}
public int getWidth() {
return width;
}
public int getHeight() {
return height;
}
}
我有rdd RDD[NewImageMetadataRow]
我想这样保存:
myRDD.saveToCassandra(keyspace, "imagemetadatav3", SomeColumns("merchant", "productid", "url"))
这会导致以下错误:
java.lang.IllegalArgumentException: requirement failed: Columns not found in com.mridang.image.NewImageMetadataRow: [merchant, productid, url]
at scala.Predef$.require(Predef.scala:281)
at com.datastax.spark.connector.mapper.DefaultColumnMapper.columnMapForWriting(DefaultColumnMapper.scala:106)
at com.datastax.spark.connector.mapper.MappedToGettableDataConverter$$anon$1.<init>(MappedToGettableDataConverter.scala:35)
at com.datastax.spark.connector.mapper.MappedToGettableDataConverter$.apply(MappedToGettableDataConverter.scala:26)
at com.datastax.spark.connector.writer.DefaultRowWriter.<init>(DefaultRowWriter.scala:16)
at com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1.rowWriter(DefaultRowWriter.scala:30)
at com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1.rowWriter(DefaultRowWriter.scala:28)
at com.datastax.spark.connector.writer.TableWriter$.apply(TableWriter.scala:423)
at com.datastax.spark.connector.RDDFunctions.saveToCassandra(RDDFunctions.scala:35)
根据我的理解(以及糟糕的scalafoo),它似乎无法从javabean中推断属性名。
另一个问题是,我的表中的列名都是小写的,删除了空格和连字符,即getter对应的cassandra列 getProductId
是 productid
.
(如果我用的是Jackson,我可以简单地加上 JsonProperty
注解。我想知道我能用卡萨德拉Map绘制器做同样的事情吗
1条答案
按热度按时间nfzehxib1#
这需要一点戳周围,但结果是这样的:
bean中的字段需要是公共的,并用
@CqlName
注解。