licode码流处理流程

RTCP

IncomingStatsHandler

分层过滤（暂时不清楚）可能会引发订阅端无视频

RtpTrackMuteHandler

发送方向带宽统计

PliPacerHandler

QualityFilterHandler

AllocationSequence

UDPPort* udp_port_; std::vector relay_ports_;std::unique_ptr udp_socket_; rtc::Network* network_;

audio_sink_-deliverAudioDatavideo_sink_-deliverVideoData

Transport::onPacketReceived

SRTP解密

OUTBOUND开始

幻灯片模式（暂时屏蔽）

IceConnectionListener

通道静默，只在OUT方向做处理

处理RTCP SR

逐层带宽统计

只在OUT方向做处理

暂时屏蔽

PacketTransportInterface

virtual rtc::PacketTransportInternal* GetInternal() = 0;

UDPPort

StunRequestManager requests_; rtc::AsyncPacketSocket* socket_; int error_;

MediaStream::sendPacketAsync

MediaStream::read

SenderBandwidthEstimationHandler

IceTransportInternal

RtpSlideShowHandler

PacketCodecParserread

PeerConnectionFactory

std::unique_ptr channel_manager_;std::unique_ptr media_engine_;std::unique_ptr call_factory_;

findontitle()creat()destory()find()check()update()reserve()

Connection

Port* port_; size_t local_candidate_index_; Candidate remote_candidate_;ConnectionInfo stats_;

RtpTransportInternal

复用ice select pair的元组发送数据前提是ICE穿透完成

PacketReader

media_stream-onTransportData

P2PTransportChannel

std::string transport_name_; int component_; PortAllocator* allocator_;std::vectorstd::unique_ptr allocator_sessions_;std::vector ports_;std::vector connections_; std::set pinged_connections_; std::set unpinged_connections_; Connection* selected_connection_ = nullptr; std::vector remote_candidates_;

SRTP加密

SRPacketHandler

1.n

RTP、RTCP码流状态统计

RtcpProcessorHandler

nicecon-onData

DtlsTransport

IceTransportInternal* ice_transport() override;

TransportListener

INBOUND开始

WebRtcConnection::write

INBOUND结束

DtlsTransport:onIceData

Transport

std::shared_ptr ice_; MediaType mediaType; std::string transport_name;std::weak_ptr transport_listener_;

RtpRetransmissionHandler

RtpPaddingGeneratorHandler

BandwidthEstimationHandler

RtpPaddingRemovalHandler

1.1

RTP

下行码流状态统计

未做处理

OneToManyProcessor

OneToManyProcessor::deliverAudioData_OneToManyProcessor::deliverVideoData_

下行丢包重传，处理RTCP 205PacketBufferService包缓存，音视频最多缓存256个包

MediaStream::onTransportData

Network

std::vector ips_;std::string name_;

IPAddress GetBestIP() const;

VP8、VP9、H264分层编码

RTCP SR报文

JsepTransport

音视频通道静默控制可能引发订阅端无音频或无视频

nice_agent_send

PacketWriter

OutgoingStatsHandler

工作流结束进入分发逻辑

BasicPortAllocatorSession

AllocationSequence* sequence_ = nullptr;Port* port_ = nullptr;

PacketTransportInternal

MediaStream::write

WebRtcConnection异步发送

subscribers-deliverAudioDatasubscribers-deliverVideoDatasubscriber(MediaStream)

RTCP 201、205、206处理，RTP透传

工作流结束进入DTLS加密和ICE SendData过程

WebRtcConnection:onTransportData

下行带宽估计

I帧平滑，采用定时任务发送I帧（暂时屏蔽）

缓存RTP包，判断是否发送RTCP RR和NACK包

port

RtpTransport

rtcp_mux_enabled_ rtc::PacketTransportInternal* rtp_packet_transport_ = nullptr;  rtc::PacketTransportInternal* rtcp_packet_transport_ = nullptr;  bool ready_to_send_ = false;  bool rtp_ready_to_send_ = false;  bool rtcp_ready_to_send_ = false;

Transport::writeOnIce

struct ConnectionInfo

bool best_connection; Candidate local_candidate; Candidate remote_candidate; uint64_t priority;

调用libnice senddata发送数据

RtcpFeedbackGenerationHandler

StunPort

void PrepareAddress() override;

解除请求I帧定时任务(200ms)，直到收到I帧请求控制

仅做p_type映射不解码

pipeline_-write

SrtpTransport

std::unique_ptr send_session_; std::unique_ptr recv_session_;rtc::Optional send_params_; rtc::Optional recv_params_;

CompositeMediaEngine/WebRtcVideoEngine/webRtcVoiceEngin

SrtpTransportInterface

virtual RTCError SetSrtpSendKey(const cricket::CryptoParams& params)virtual RTCError SetSrtpReceiveKey(const cricket::CryptoParams& params) = 0;};

仅对RTCP 201、205、206处理，带宽估计

DtlsTransportInternal

virtual IceTransportInternal* ice_transport() = 0;

pipeline_-read(

LayerDetectorHandlerread

PortAllocator

int min_port_; int max_port_;typedef std::set ServerAddresses;ServerAddresses stun_servers_; std::vector turn_servers_; int candidate_pool_size_ = 0; // Last value passed into SetConfiguration. std::dequestd::unique_ptr pooled_sessions_; webrtc::TurnCustomizer* turn_customizer_ = nullptr;

virtual void StartGettingPorts() = 0;

对视频且RTP类型为RED_90000做FEC

下行包缓存，用于丢包重传，每个Stream有独立的缓存队列

WebRtcConnection

RtpTransportInterface

virtual PacketTransportInterface* GetRtpPacketTransport() const = 0;virtual PacketTransportInterface* GetRtcpPacketTransport() const = 0;virtual RTCError SetParameters(const RtpTransportParameters& parameters) = 0; virtual RtpTransportParameters GetParameters() const = 0;

Candidate

rtc::SocketAddress address_;rtc::SocketAddress related_address_;std::string transport_name_; uint16_t network_id_; uint16_t network_cost_;std::string transport_name_;uint32_t generation_;

// The name of the transport channel of this candidate. // TODO(phoglund): remove. const std::string& transport_name() const { return transport_name_; }

LayerBitrateCalculationHandler

不做任何处理

FecReceiverHandler

DtlsTransport::write

OUTBOUND结束

JsepTransportController

SFU

工作流开始