Kafka生产者类图
2021-12-09 20:45:23 0 举报
Kafka生产者类图
作者其他创作
大纲/内容
Builder
RecordAccumulator
+append()+drain()
-client
ClientRequest
ProducerBatch
-sender
sender线程
add
ProducerInterceptors
+onSend()+onAcknowledgement()
使用KafkaProducer客户端发送消息(ProducerRecord),需要经历拦截器(Interceptor)、序列化器(Serializer)和分区器(Partitioner)等一系列的作用
ProducerInterceptor
-accumulator
request
drain
main线程
ProduceRequest
send
interfacePartitioner
+partition()
-interceptors
Deque
一条消息(ProducerRecord)在经过拦截器、序列化器、分区器处理之后,被存入到RecordAccumulator中。在RecordAccumulator中维护了一个名为batches的Map,其以分区信息(TopicPartition)为键,双端队列(Deque)为值,这个队列由若干ProducerBatch组成,而一个ProducerBatch又包含若干条消息
-clientId:String-producerConfig:ProducerConfig
+KafkaProducer()+send(ProducerRecord)
interfaceRunnable
+run()
以broker节点为维度,存放发往broker节点的封装后的InFlightRequest消息,根据配置(max.in.flight.requests.per.connection)可能限制每个broker的缓存请求
-inFlightRequests
a:TopicPartitiona IN node1
InFlightRequests
+add()+canSendMore():boolean
sender线程在实例化KafkaProducer时被创建,并以守护线程的方式运行
进一步封装消息
c:TopicPartitiona IN node1
interfaceSerializer
+serialize()+configure()
b:TopicPartitiona IN node2
append
将从RecordAccumulator获取的消息集合封装
-partitioner
-keySerializer
-valueSerializer
InFlightRequest
destination:Stringheader:RequestHeadercallback:RequestCompletionHandler
+InFlightRequest()+completed()+disconnected()
NetworkClient
-selector:Selectable
AbstractRequest
0...n
Sender
+Sender()+run()-runOnce()-sendProducerData()-sendProduceRequests()-sendProduceRequest()
0 条评论
下一页