spark on yarn启动流程
2017-03-01 21:41:35 0 举报
Spark on YARN启动流程如下:首先,客户端向ResourceManager申请启动ApplicationMaster,然后ResourceManager分配container,并通知NodeManager启动container。接着,ApplicationMaster与NodeManager通信,要求分配资源并启动Executor。Executor运行任务后,将结果反馈给ApplicationMaster。最后,ApplicationMaster将结果汇总后返回给客户端。
作者其他创作
大纲/内容
4
ClientRMService
YarnClient
封装ApplicationSubmissionContext
client
NodeManager
ApplicationMaster
1
executor
createApplication
异步执行开始
3
ResourceManager
2
ApplicationId
YarnClientApplication
Client
申请一个ApplicationId
submitApplication
0 条评论
回复 删除
下一页