Skip to content

chore(audit): audit BitAndAgg and expand tests#4437

Open
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:autonomous-audit/bit_and
Open

chore(audit): audit BitAndAgg and expand tests#4437
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:autonomous-audit/bit_and

Conversation

@andygrove
Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A. Autonomous audit pass.

Rationale for this change

Audit of the bit_and expression (BitAndAgg) against Spark 3.4.3, 3.5.8, and 4.0.1. The Spark source is identical across all three versions, and Comet delegates the aggregate to DataFusion's bit_and_udaf. Static review found no behavioural divergence and no missing getIncompatibleReasons/getUnsupportedReasons overrides. Existing SQL coverage only exercised int with a single nullable row; this audit expands coverage to the remaining integral types and additional edge cases.

What changes are included in this PR?

  • Audit sub-bullets in spark_expressions_support.md for Spark 3.4.3, 3.5.8, 4.0.1
  • Expanded expressions/aggregate/bit_agg.sql: all integral types (tinyint, smallint, int, bigint), all-NULL group, empty input, HAVING clause, DISTINCT, boundary values (Int.MinValue/MaxValue, Long.MinValue/MaxValue), and a literal argument

How are these changes tested?

  • ./mvnw test -DwildcardSuites=CometSqlFileTestSuite -Dsuites="org.apache.comet.CometSqlFileTestSuite bit_agg" -Dtest=none -Dscalastyle.skip=true (passes locally)

Note: the audit also drafted a bit_and OVER (...) window-form test, but removed it because Comet's WindowExec is gated behind spark.comet.operator.WindowExec.allowIncompatible=true due to known correctness issues unrelated to bit_and. That fallback path is not specific to this expression.

Scaffolded by the audit-comet-expression-autonomous skill.

@andygrove andygrove marked this pull request as ready for review May 27, 2026 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant