linkis执行逻辑架构
2020-07-21 10:12:02 0 举报Linkis是一个打通了多个计算存储引擎如：Spark、TiSpark、Hive、Python和HBase等，对外提供统一REST/WebSocket/JDBC接口，提交执行SQL、Pyspark、HiveQL、Scala等脚本的计算中间件。
作者其他创作
大纲/内容
EngineGroupFactory
submit
ParallelScheduler
SparkSingleEntranceEngine
FIFOUserConsumer
创建
ServerListenerEventBus
SingleEngineSelector
askExecutor
开始Job异步执行
EngineHook
EntranceExecutionJob
心跳
EmMetaDataDao
FIFOScheduler
请求Engine
FIFOConsumerManager
EntranceGroupFactory
EngineExecutorManagerImpl
DefaultRMContext
EngineManagerSpringConfiguration
SparkCommonProcessEngine
Parse成RequestPersistTask和EntranceExecutionJob
ResponserNewEngine/ResponseNewEnggineStatus
EngineContextImpl
SparkCommonProcessBuilder
SparkEngineExecutorFactory
EntranceSpringConfiguration
EngineParser
ResourceRequesterImpl
ModuleResourceRecordService
EventSchedulerImpl
requestResource
SparkSqlExecutor
实例化注册Listener
EntranceWebSocketService.onEvent
DefaultEntranceContext
EntranceEventListenerBus
SparkEngineExecutor
PostConstruct
EngineSchedulerContextImpl
CacheLogManager
进程一个Yarn client模式的Spark-submit class是DataWorkCloudEngineApplication
ResourceLockDao
2.create
EngineManagerImpl
request
启动进程
SparkPythonExecutor
EmResourceMetaDataDao
Entrance
EntranceExecutionJob.run
UserResourceRecordService
SparkEngineBuilder
ServerSocket.onWebSocketText
EngineReciver
SparkScalaExecutor
SparkEngineCreator
DefaultModuleResourceManager
CommonEngineJob
ResourceLockService
DataWorkCloudApplication
setupControllerServer
DefaultEntranceServer.execute
DefaultUserResourceManager
ParallelConsumerManager
FIFOGroup
QueryPersistenceEngine
CommonEntranceParser
实例化
EngineRequesterImpl
start
SparkEngineManagerSpringConfiguration
CommonEngineJob.run
EntranceExecutorManagerImpl
DefaultResourceManager
buildEngine
获取并锁定资源
CodeParser
ResourceManager
EntranceResultSetEngine
ModuleInfo
DefaultUserMetaData
1.request
UserResourceMetaDataDao
createExecutor
BDPJettyServerHelper
EngineServerSpringConfiguration
execute
requestEngine
EngineManager
ResourceManagerClient
EngineManagerReceiver
EntranceSchedulerContext
EngineFactoryImpl
RMReceiver
注入
EngineManagerContextImpl
SparkEngineResourceFactory
EngineServerImpl
3.异步
Engine
RequestTaskExecute
监控进程
0 条评论
下一页
