sparkStreaming 深度理解
2021-07-19 14:51:52 0 举报
sparkStreaming架构深入浅出 数据简单可视流图
作者其他创作
大纲/内容
ForEachDStream()
task4
pear banana orangepotato orange apple......
......
kafka集群
task5
⑤reduceBykey
task1
Executor2(3 core)
Executor1(3 core)
DirectKafkaInputDStream(KafkaRDD)
......
FaltMappedDStream()
②KafkaUtils.createDirectStream
task3
task2
Spark Streaming集群(简称SS)
......
①
RDD
③foreachRDD
orange pear orangeorange banana potato......
driver
ShuffledDStream()
⑦-1
⑥foreachRDD
数据流
apple pear orangeorange banana potato......
MappedDStream()
④Map处理
⑧SampleHbaseClient.getConnection()
⑦foreachPartition
BlockManager
③flatMap处理
banana pear orangepotato orange apple......
potato apple orange pear orange banana......
<orange pear orange><orange banana potato>......
kafka集群topic:songTopicpartitions:5
<apple pear orange><orange banana potato>......
0 条评论
下一页