fix: preserve total_byte_size in calculate_total_byte_size when num_r…

[bert-beyondloops]

Which issue does this PR close?

• Closes Statistics::calculate_total_byte_size discards a known total_byte_size when num_rows is unknown #24026

Rationale for this change

Statistics::calculate_total_byte_size is meant to derive total_byte_size from num_rows and the schema's fixed-width columns. When all columns have a primitive width but num_rows is Precision::Absent, the old code computed
self.num_rows.multiply(&Precision::Exact(size)), and Precision::multiply returns Precision::Absent whenever either operand is Absent. This silently overwrote any previously known total_byte_size (exact or inexact) with
Absent, even though the non-primitive-width branch already handled this situation correctly by downgrading the existing value to inexact instead of discarding it.

What changes are included in this PR?

• In Statistics::calculate_total_byte_size, when the schema is all fixed-width but num_rows is Precision::Absent, keep the existing total_byte_size and downgrade it to inexact via to_inexact(), instead of overwriting it with
  Absent.
• Updated the doc comment on calculate_total_byte_size to describe this behavior.
• Added test_calculate_total_byte_size covering: an all-primitive schema with known row count (exact size), an all-primitive schema with unknown row count (preserved but downgraded to inexact), and a non-primitive schema
  (always downgraded to inexact regardless of row count).

Are these changes tested?

Yes — added `stats::tests::test_calculate_total_byte_size, exercising all three branches of the updated match.

Are there any user-facing changes?

No public API changes. Statistics propagation is more accurate (previously known total_byte_size estimates are no longer dropped to Absent when num_rows is unknown), which may result in slightly better cost-based planning
decisions in some cases.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.04%. Comparing base (dbcb5c0) to head (8dc00e2).
⚠️ Report is 57 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24027      +/-   ##
==========================================
+ Coverage   80.85%   81.04%   +0.18%     
==========================================
  Files        1101     1105       +4     
  Lines      374933   380110    +5177     
  Branches   374933   380110    +5177     
==========================================
+ Hits       303166   308052    +4886     
- Misses      53671    53838     +167     
- Partials    18096    18220     +124     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pepijnve]

This change makes sense to me since it avoids downgrading Exact total_byte_size to absent when num_rows is Absent.

[asolimando]

Intuitively this makes sense to me too, hopefully we will soon be able to have a more data-driven approach thanks to #23975.

[asolimando]

LGTM

[AdamGS]

This looks good to me overall, just one small test comment.

I'm not sure I fully understand the existing behavior that always downgrades non-primitive schemas to inexact, but I also recall that it doesn't make a huge difference.

[asolimando]

This looks good to me overall, just one small test comment.

I'm not sure I fully understand the existing behavior that always downgrades non-primitive schemas to inexact, but I also recall that it doesn't make a huge difference.

Hey Adam, my understanding is that non-primitive types (composite types, arrays, list, maps) contribute a varying amount of bytes, depending on the size of the collection, which cannot be really known, so it's inexact by nature.

Primitive types, on the contrary, are tied to a fixed byte-size which can be computed exactly (so given a schema composed only by primitive types, you can tell how many bytes a single row takes, and total byte size is row byte size times number of rows).

[AdamGS]

That's the part I understand, but say we have a schema with a single string column and the table provider (or any other source) reports exact size and and exact row count, my understanding is that the current behavior (even before this PR) will always set it to Inexact.

[asolimando]

That's the part I understand, but say we have a schema with a single string column and the table provider (or any other source) reports exact size and and exact row count, my understanding is that the current behavior (even before this PR) will always set it to Inexact.

OK, I see what you mean.

The only caller of this function today is projection.rs#L724: if we used byte_size from ColumnStatistics instead of DataType::primitive_width(), I think we could keep exact (provided that the column-level byte sizes are exact too), even for non-primitive types (length varying types).

This of course depends on the projection expressions, if they are pure input column references (dropping some columns, for instance), and that columns have exact byte size values, it would be doable.

When you say it doesn't matter much in practice I guess it's because Parquet sets Inexact for non-primitive data types.

If you agree on my reading and it feels useful, I can file a follow-up issue to track this.

[bert-beyondloops]

🙏 @AdamGS @asolimando @pepijnve