avro模式和可选记录

mhd8tkvw  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(382)

大家好,我需要为下面的示例创建avro模式;

{ "Car" : { "Make" : "Ford" , "Year": 1990 , "Engine" : "V8" , "VIN" : "123123123" , "Plate" : "XXTT9O", 
"Accident" : { "Date" :"2020/02/02" , "Location" : "NJ" , "Driver" : "Joe" } ,
"Owner" :  { "Name" : "Joe" , "LastName" : "Doe" } }

意外和所有者是可选对象,创建的模式还需要验证以下子集消息;

{ "Car" : { "Make" : "Tesla" , "Year": 2020 , "Engine" : "4ELEC" , "VIN" : "54545426" , "Plate" : "TESLA" }

我阅读了avro规范,看到了很多可选的属性和数组示例,但没有一个是有效的。如何将记录定义为可选记录?谢谢。
没有任何可选属性的模式正在工作。

{
  "name": "MyClass",  "type": "record",  "namespace": "com.acme.avro",  "fields": [
    {
      "name": "Car",   "type": {
        "name": "Car","type": "record","fields": [
          { "name": "Make",    "type": "string"   },
          { "name": "Year",    "type": "int"      },
          { "name": "Engine",    "type": "string" },
          { "name": "VIN",    "type": "string"    },
          { "name": "Plate",    "type": "string"  },
          { "name": "Accident",
                "type":
                { "name": "Accident",
                  "type": "record",
                  "fields": [
                       { "name": "Date","type": "string" },
                       { "name": "Location","type": "string" },
                       { "name": "Driver", "type": "string" }
                    ]
                 }
          },

          { "name": "Owner",
            "type":
                {"name": "Owner",
                 "type": "record",
                 "fields": [
                {"name": "Name",  "type": "string" },
                {"name": "LastName", "type": "string" }
              ]
            }
          }
        ]
      }
    }
  ]
}

当我按照建议更改所有者对象时,avro工具返回错误。

{ "name": "Owner",
        "type": [
            "null",
            "record" : {
              "name": "Owner",
              "fields": [
                 {"name": "Name",  "type": "string" },
                 {"name": "LastName", "type": "string" }
               ]
             }
          ] , "default": null  }
       ]
   }
}

] }
测试:

Projects/avro_test$ java -jar avro-tools-1.8.2.jar fromjson --schema-file CarStackOver.avsc Car.json > o2
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" org.apache.avro.SchemaParseException: org.codehaus.jackson.JsonParseException: Unexpected character (':' (code 58)): was expecting comma to separate ARRAY entries
 at [Source: org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4034c28c; line: 26, column: 13]
        at org.apache.avro.Schema$Parser.parse(Schema.java:1034)
        at org.apache.avro.Schema$Parser.parse(Schema.java:1004)
        at org.apache.avro.tool.Util.parseSchemaFromFS(Util.java:165)
        at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:83)
        at org.apache.avro.tool.Main.run(Main.java:87)
        at org.apache.avro.tool.Main.main(Main.java:76)
Caused by: org.codehaus.jackson.JsonParseException: Unexpected character (':' (code 58)): was expecting comma to separate ARRAY entries
 at [Source: org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4034c28c; line: 26, column: 13]
        at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433)
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521)
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:442)
        at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:482)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:222)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:197)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
        at org.codehaus.jackson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
        at org.codehaus.jackson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
        at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
        at org.codehaus.jackson.map.ObjectMapper.readTree(ObjectMapper.java:1344)
        at org.apache.avro.Schema$Parser.parse(Schema.java:1032)
mzillmmw

mzillmmw1#

您可以通过与 null .
这样地:

{
    "name": "Owner",
    "type": [
      "null",
      {
        "name": "Owner",
        "type": "record",
        "fields": [
          { "name": "Name", type": "string" },
          { "name": "LastName", type": "string" },
        ]

      }
    ],
    "default": null
  },

相关问题