UML-ValueReader
2017-11-18 20:00:55 16 举报
Parquet index design
作者其他创作
大纲/内容
ParquetFileReader
BigPackingValueseReader
FixedLenByteArrayPlainValuesReader
PageReader
readDictionaryPage():DictionaryPagegetTotalValueCount():longreadPage:DataPage
IndexValuesReader
BooleanPlainValuesReader
ColumnReadStoreImpl
+ pageReadStore:PageReadStore+ recordConverter:GroupConverter+ schema MessageTypewriterVersion:ParsedVersion
DeltaLengthByteArrayValuesReader
DictionaryValuesReader
ValuesReader
PageReadStore
getPageReader(descriptor:ColumnDescriptor)getRowCount():long
ColumnReadStore
+ getColumnWriter(ColumnDescriptor():ColumnWriter+ flush()+ engRecord()+ getAllocatedSize()+ getBufferedSize()+ memUsageString()
ColumnReader
compressedIndexPage:IndexPage
ColumnChunkPageReadStore
DeltaByteArrayReader
ColumnChuknPageReader
- decompressor:BytesDecompressor- valueCount:long- compressedPages:List- compressedDictionaryPage:DictionaryPage-
+ getTotalValueCount():long+ readPage():dataPage+ readDictionaryPage():DictinaryPage+
BoundedIntValuesReader
DataPage
- valueCount:int
+ getValueCount:int+ accept(visitor:Visitor):
BynaryPlainValuesReader
ZeroIntegerValuesReader
PlainValuesReader
IndexPage
readIndexPage():IndexPage
PlainValuesDictionary
index:Index
DictionaryPage
- bytes:BytesInput- dictionarySize:int- encoding:Encoding
+ getBytes():Bytes+ getDictionarySize():int+ getEncoding:()Encodingcopy():DictionaryPage
ByteBigPackingValuesReader
DeltaBinaryPackingValuesReader
DataPageV2
- rowCount:int- nullCount:int- repetitionLevels:BytesInput- definitionLevels:BytesInput- dataEncoding:Encoding- data:BytesInput- statistics:Statistics- isCompressed:boolean
+ accept(visitor:Visitor): T
Page
- compressedSize:int- uncompressedSize:int
+ getCompressedSize():int+ getUncompressedSize():int
InternalParquetRecordReader
- columnlOFactory: ColumnlOFactory = null- filter: Filter- requestedSchema: MessageType- fileSchema: MessageType- columnCount: int- readSupport: ReadSupport- recordConverter: RecordMaterializer- currentValue: T- total: long- current: long = 0- currentBlock: int = -1- reader: ParquetFileReader- recordReader: RecordReader- strictTypeChecking: boolean- totaLTimeSpentReadingBytes: Long- totalTimeSpentProcessingRecords: long- startedAssemblingCurrentBlockAt: long- totalCountLoadedSoFar: long = 0- file: Path- unmaterializableRecordCounter: UnmaterializableRecordCounter
ColumnReaderImpl
writerVersion:ParsedVersionpath:ColumnDescriptortotalValueColunt:longpageReader:PageReaderdictionary:DictionaryrepetitionLevalColumn:IntIteratordefinitionLevelColumn:IntIteratordataColumn:valuesReadercurrentEncoding:EncodingrepetitionLevel:intdefinitionLevel:intdictionaryId:intengOfPageValueCount:long:longreadValues:intpageValueCount:intconverter:PrimitiveConverterbinding:BindingvalueRead:boolean
RunLengthBitPackingHybridValuesReader
收藏
0 条评论
回复 删除
下一页