如何在java中将值Map到嵌套的avro模式

tez616oj  于 2021-08-20  发布在  Java
关注(0)|答案(0)|浏览(301)

我试图为Kafka主题提供数据。作为需求的一部分,我从数据库中提取数据并将其Map到avro模式。我知道如何解析数据,因为模式很简单,如下所示:
简单avro模式:

  1. {
  2. "type":"record",
  3. "namespace":"demo",
  4. "name":"coreloader330",
  5. "doc":"sample avro schema file for core loader 330",
  6. "fields":[
  7. { "name" : "HEALTH_PLAN", "type" : ["null", "string"]}
  8. ]
  9. }

通用生成器java:

  1. public GenericRecordBuilder getGenericBuilder() {
  2. GenericRecordBuilder builder = null;
  3. Schema.Parser parser = new Schema.Parser();
  4. try {
  5. Schema schema = parser.parse(new File("src/main/resources/coreloader.avsc"));
  6. builder = new GenericRecordBuilder(schema);
  7. } catch (IOException e) {
  8. e.printStackTrace();
  9. }
  10. return builder;

将值Map到生成器:

  1. public void produceMessage(){
  2. //Create Producer
  3. KafkaProducer<String, GenericRecord> producer = new KafkaProducer<String, GenericRecord>(props);
  4. try{
  5. while(rs.next()){
  6. // System.out.println(rs.getString("HEALTH_PLAN"));
  7. builder.set("HEALTH_PLAN", rs.getString("HEALTH_PLAN")); ---> mapping values to avro schema field
  8. GenericData.Record data = builder.build();
  9. ProducerRecord<String, GenericRecord> record = new ProducerRecord<String, GenericRecord>(topic, data);
  10. producer.send(record);
  11. producer.flush();
  12. }

但是,如果模式很复杂,如何Map值,如何到达Map的嵌套字段?
假设我有下面这样的模式,如何遍历并将值Map到sourcesystem字段?

  1. "type": "record",
  2. "name": "demo",
  3. "namespace": "example",
  4. "fields": [
  5. {
  6. "name": "data",
  7. "type": {
  8. "type": "record",
  9. "name": "Data",
  10. "fields": [
  11. {
  12. "name": "SourceSystem",
  13. "type": [
  14. "null",
  15. "string"
  16. ],
  17. "default": null
  18. }
  19. ]
  20. }
  21. }

在下面的代码中,我的Map应该是什么?

  1. builder.set(<?>, rs.getString("source system")); ---> mapping values to avro schema field

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题