Skip to content

chore(audit): audit any_value and expand tests#4438

Open
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:autonomous-audit/any_value
Open

chore(audit): audit any_value and expand tests#4438
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:autonomous-audit/any_value

Conversation

@andygrove
Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A. Autonomous audit pass.

Rationale for this change

Audit of the any_value expression against Spark 3.4.3, 3.5.8, and 4.0.1.

AnyValue is a RuntimeReplaceableAggregate whose replacement is First(child, ignoreNulls), and the optimizer rule ReplaceExpressions rewrites it before physical planning. Comet therefore handles any_value transparently through CometFirst; no dedicated serde exists. The class is identical across the three Spark versions checked. Coverage in first_last.sql is comprehensive for first, but no test in the repo exercises the any_value name directly. A regression in the analyzer rewrite, in a future AnyValue.replacement, or in Comet's plan conversion would silently bypass that coverage.

What changes are included in this PR?

  • Audit sub-bullets in spark_expressions_support.md for any_value across the three Spark versions.
  • New SQL test file spark/src/test/resources/sql-tests/expressions/aggregate/any_value.sql exercising any_value directly: basic, with grouping, isIgnoreNull literal forms, all-null groups, mixing with other aggregates, and equivalence to first on identical inputs.

How are these changes tested?

./mvnw test -Dsuites="org.apache.comet.CometSqlFileTestSuite any_value" -Dtest=none passes locally (Spark 3.5 profile, 1 test, 0 failures).

Scaffolded by the audit-comet-expression-autonomous skill.

@andygrove andygrove marked this pull request as ready for review May 27, 2026 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant