首页  思维导图  详情

Flink uid和uidHash

2021-05-25 23:49:09   0  举报





AI智能生成

flink 源码

flink 数据分析

作者其他创作

大纲/内容

uid

设置org.apache.flink.api.dag.Transformation#setUid

uidHash

设置org.apache.flink.api.dag.Transformation#setUidHash

设置
org.apache.flink.streaming.api.graph.
StreamGraphGenerator#transform

if (transform.getUid() != null) {
streamGraph.setTransformationUID(transform.getId(), transform.getUid());
}
if (transform.getUserProvidedNodeHash() != null) {
streamGraph.setTransformationUserHash(transform.getId(), transform.getUserProvidedNodeHash());
}

org.apache.flink.streaming.api.graph.
StreamingJobGraphGenerator#createJobGraph()
setChaining(hashes, legacyHashes, chainedOperatorHashes);

defaultStreamGraphHasher
生成hashes

StreamGraphHasherV2

generateNodeHash
判断用户是否设置uid
generateDeterministicHash
or
generateUserSpecifiedHash

* Generates a deterministic hash from node-local properties and input and output edges.
有可能重复

* Generates a hash from a user-specified ID.

legacyStreamGraphHashers
生成legacyHashes

StreamGraphUserHashHasher
使用用户提供的uid hash

traverseStreamGraphAndGenerateHashes

org.apache.flink.streaming.api.graph.
StreamingJobGraphGenerator#createJobVertex

new JobVertexID(hash)

legacyJobVertexIds