PARQUET-2171: (followup) add read metrics and hadoop conf integration for vector io reader#1330
Conversation
… for vector io reader
|
@wgtmac, @steveloughran Some minor additions to the vector io based file reader. Adds the read metrics added in the serial reader path. Also adds the default construction in read options to read the hadoop conf for the vector io setting. |
|
Thank you @wgtmac ! |
|
looks great. If there's another 14.0 RC, will this go in to it? Note we create lots and lots of IOstatistics, for vector reads we include #of bytes read and discarded along with all the other timings. My WiP to make that accessible via reflection will help, but it'd still need work in parquet to aggregate. |
|
I think this is already included in the 1.14.0 RC0/RC1 |
… for vector io reader (apache#1330)
This is a followup with minor fixes/additions for the vector io based file reader
Jira
Tests
Documentation
Existing documentation is sufficient