AWS data pipeline
2023-01-06 11:03:18 0 举报利用AWS云框架进行数据管道构建
作者其他创作
大纲/内容
4
Wide Table As the Final Result of ETL
Lambda
2
Feature-Data-Zone
Glue Service
Glue-Crawler
Glue-JobFeature-Extractor
Event-Trigger
Schedule-Trigger
Provide Raw Data
1
Crawling Raw Data from DataLake into Glue-Database
PredictionsResult
5
3
S3-Data scource
Curated-Data-Zone
Event-TriggerCrawler-Trigger-GlueETLJob
Raw-Data-Zone
Extracting Raw Data from Data Source into DataLake
Event-TriggerS3-Trigger-Crawler
Business Intelligence
Data Pipeline Based on AWS
XGBoost
products.csvdepartment.csvaisle.csv
Event-TriggerS3-Trigger-SageMaker
Database
user_feature_1user_feature_2up_featureprd_feature
Recommendation-Data-Zone
Data-Lake
Delivery
Orders.csvdata_order_priordata_order_train
Glue-JobWideTable for ML
SageMaker reads the Wide table as train dataset and feedback the ML model
收藏
0 条评论
下一页
