如何将list对象转换为javadstream spark?

dhxwm5r4  于 2021-05-27  发布在  Spark
关注(0)|答案(1)|浏览(645)

我有以下清单:

  1. List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> rddQueue =
  2. Arrays.asList(tupleList);

如何将其转换为javadstream spark?提前谢谢!

2izufjch

2izufjch1#

你可以用 JavaStreamingContext 和一个 Queue<T> :

  1. List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>
  2. list = Arrays.asList();
  3. // Random batch time, for the sake of the example.
  4. JavaStreamingContext jssc =
  5. new JavaStreamingContext("local[*]", "TestApp", new Duration(4000));
  6. Queue<JavaRDD<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>> rddQueue =
  7. new LinkedList<>();
  8. rddQueue.add(jssc.sparkContext().parallelize(list));
  9. JavaDStream<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> dStream =
  10. jssc.queueStream(rddQueue);

相关问题