如何解决spark从elasticsearch读取数据时多重Map中发现不兼容类型的问题?

ht4b089n  于 2021-05-24  发布在  Spark
关注(0)|答案(0)|浏览(641)

如何绕过此问题(无需重新编制索引):

  1. Py4JJavaError: An error occurred while calling o2649.load.
  2. : org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Incompatible types found in multi-mapping: Field [metadata.cscope.0] has conflicting types of [LONG] and [OBJECT].

我尝试通过es.read.field.exclude排除此字段,但没有帮助。
我的阅读:

  1. reader = spark.read.format("org.elasticsearch.spark.sql") \
  2. .option("es.port", "9200") \
  3. .option("spark.es.net.http.auth.user", "") \
  4. .option("spark.es.net.http.auth.pass", "") \
  5. .option("es.nodes.wan.only", "true") \
  6. .option("es.mapping.date.rich", "false") \
  7. .option("es.http.timeout", "3m") \
  8. .option("es.read.field.include", "log") \
  9. .option("es.read.field.exclude", "metadata*") \
  10. .option("es.read.metadata", "false") \
  11. .option("es.nodes", IP)
  12. df = reader.load(INDEX_NAME)

Spark塞。2.4.5库:elasticsearch-spark-20_2.11-7.6.2.jar

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题