Skip to content

Python SqlTransform fails with an unhelpful Java error when type inference uses the pythonsdk_any fallback #20738

@damccorm

Description

@damccorm

See https://stackoverflow.com/questions/65903919/local-testing-apache-beam-with-sqltransform-in-python-sdk-i-receive-an-error for a good example of this.

We should throw a more helpful error in this situation:

  • It should mention the problematic field(s)
  • It should suggest a solution (try wrapping in casts if using beam.Row, don't use Any in a NamedTuple)
  • It might point to some documentation about supported SQL types.

Imported from Jira BEAM-11690. Original Jira may contain additional context.
Reported by: bhulette.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions