首页  流程图  详情

Parquet

2018-12-11 11:09:19   1  举报





为你推荐

查看更多



LarVar构建流程图

碎片文件处理

Data Lake Architecture

大数据测试

数据湖-数据流向图

（90%）大数据文件格式梳理：Parquet、Avro、OR

Parquet数据结构示意图

parquet

作者其他创作

大纲/内容

values

Footer length(4 bytes)

Page 0

Magic Number(4 bytes): \"PAR1\"

Column b meta data- type/path/encoding/codec- num values- offset of first data page- offset of first index page- compressed/uncompressed size- extra key/value pairs

Magic Number(4 bytes)：\"PAR1\"

Repetition levels

Definition levels

Column a meta data- type/path/encoding/codec- num values- offset of first data page- offset of first index page- compressed/uncompressed size- extra key/value pairs

Row group 0

Parquet File

Column c meta data

Column b

Row group 1

Page header(ThriftCompactProtocol)

Page 1

Footer

Column a

- Version- Schema- extra key/value paris

FileMetaData(ThriftCompactProtocol)