scala—spark结构化流中Dataframe的访问单元值

nfs0ujit  于 2021-07-13  发布在  Spark
关注(0)|答案(0)|浏览(277)

我想访问结构化流Dataframe的单元格值,将其传递给impala
代码:

val CANDIDATESWABTEST_df: DataFrame = spark.readStream
  .format("kafka")
  .option("kafka.bootstrap.servers", "xxxx:9092")
  .option("subscribe", "xxxx")
  .option("group.id", "tests")
  .option("startingOffsets", "latest")
  .option("failOnDataLoss", false)
  .load()
val parsed_CANDIDATESWABTEST_df: DataFrame = CANDIDATESWABTEST_df
  .select(from_json(expr("cast(value as string) as actualValue"), schemaCANDIDATESWABTEST).as("s")).select("s.*").as("dftests")

尝试#1

var querySQL = "select * from v_candidate_info where candidate_id=" +parsed_CANDIDATESWABTEST_df.select("CANDIDATEID").take(0) +""

尝试#2

var querySQL = "select * from v_candidate_info where candidate_id=" +parsed_CANDIDATESWABTEST_df.select("CANDIDATEID")+""

根据链接尝试#3

.as[String].collect()

所有这些尝试都没有成功

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题