我有以下清单:
List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> rddQueue = Arrays.asList(tupleList);
List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> rddQueue =
Arrays.asList(tupleList);
如何将其转换为javadstream spark?提前谢谢!
2izufjch1#
你可以用 JavaStreamingContext 和一个 Queue<T> :
JavaStreamingContext
Queue<T>
List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> list = Arrays.asList();// Random batch time, for the sake of the example.JavaStreamingContext jssc = new JavaStreamingContext("local[*]", "TestApp", new Duration(4000));Queue<JavaRDD<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>> rddQueue = new LinkedList<>();rddQueue.add(jssc.sparkContext().parallelize(list));JavaDStream<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> dStream = jssc.queueStream(rddQueue);
List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>
list = Arrays.asList();
// Random batch time, for the sake of the example.
JavaStreamingContext jssc =
new JavaStreamingContext("local[*]", "TestApp", new Duration(4000));
Queue<JavaRDD<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>> rddQueue =
new LinkedList<>();
rddQueue.add(jssc.sparkContext().parallelize(list));
JavaDStream<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> dStream =
jssc.queueStream(rddQueue);
1条答案
按热度按时间2izufjch1#
你可以用
JavaStreamingContext
和一个Queue<T>
: