在scala中读取带有架构的json文件时出错

y0u0uwnf  于 2021-07-12  发布在  Spark
关注(0)|答案(1)|浏览(375)

读取arraytype值(phonenumbers)时出错,如果没有arraytype值,我可以读取rest值。

{
    "firstName": "Rack",
    "lastName": "Jackon",
    "gender": "man",
    "age": 24,
    "address": {
        "streetAddress": 126,
        "city": "San Jone",
        "state": "CA",
        "postalCode": 394221
    },
    "phoneNumbers": [
        { "type": "home", "number": 7383627627}
    ]
}

My schema ->
val schema=StructType(List(
      StructField("firstName",StringType),
      StructField("lastName",StringType),
      StructField("gender",StringType),
      StructField("age",IntegerType),
      StructField("address",StructType(List(
        StructField("streetAddress",StringType),
        StructField("city",StringType),
        StructField("state",StringType),
        StructField("postalCode",IntegerType)))),
      StructField("phoneNumbers",ArrayType(StructType(List(
      StructField("type",StringType),
      StructField("number",IntegerType))))),
    ))

json_df.selectExpr("firstName","lastName",
      "gender","age","address.streetAddress","address.city",
      "address.state","address.postalCode",
      "explode(phoneNumbers) as phone","phone.type","phone.number").drop("phone").show()

当我这么做的时候 .show ,它只显示列名,不显示值,但是当我不使用“phonenumbers”数组时,它可以正常工作。

e5njpo68

e5njpo681#

IntegerType 表示4字节有符号整数,最大值为2147483647,不能容纳电话号码。任意使用 LongType 或者 StringType 电话号码。
你的调查没有结果 select 查询,因为您正在分解电话号码的空数组,该数组返回0行。数组为空,因为电话号码无法保存在 IntegerType 列。

相关问题