RPC和反压
2021-05-10 22:37:55 2 举报flink内部通信
作者其他创作
大纲/内容
OperatorChain
PipelinedResultPartition
NettyMessageEncoder
SingleInputGate...
InputGate
ResultSubpartition
CreditBasedPartitionRequestClientHandler
TaskManager
memorySegment
StreamTask
同一个TM中直接从上游的subpartition中将数据拿出来用即可
PartitionRequestServerHandler
NettyMessageDecoder
PartitionRequestQueue
ResultPartition
写入到heap数组buffer中
netty发送消息流程
PipelinedSubpartition
CreditBasedSequenceNumberingViewReader
RecordWriter
ResultPartitionWriter
NettyShuffleEnvironment
inputChannels
LocalBufferPool
NettyPartitionRequestClient
mailboxProcessor
recordWrites
subpartitions
ResultSubpartitionView
RecordWriterOutput...
NetworkBufferPool
DataOutputSerializer
消息类型
RecordWriterOutput
streamOutputs
availabilityListener
buffer数组
LocalInputChannel
stateBackend
SerializationDelegate
RemoteInputChannel
RecordWriterDelegate
client
outputTag
networkBufferPool
OneInputStreamTask
Task
PipelinedResultPartition...
StreamTaskInput
数据copy到非heap内存中
发送消息流程
封装用户function的StreamTask
定时100ms发送数据到下游这里实际就是将非memorySegment中数据copy到netty内存发送到下游
inputProcessor
server
反压
NetworkSequenceViewReader...
allReaders
CreditBasedPartitionRequestClientHandler
通过Netty进行的连接
SingleInputGate
StreamOperatorWrapper
RecordWriter...
TM
sslhandler
ChannelSelector
0 条评论
下一页
