Skip to content

powerbi databricks parser does not support expressions #24874

@mmigdiso

Description

@mmigdiso

Affected module
Ingestion Framework

Describe the bug
When powerbi databricks queries use expressions in the native queries, the parser is not able to create the lineages

To Reproduce
in the following native query, the Name=DB needs to be resolved using the dataset expression section.
"""let
Source = Value.NativeQuery(Databricks.Catalogs(Databricks_Server, Databricks_HTTP_Path, [Catalog="DEMO_CATALOG", Database=null, EnableAutomaticProxyDiscovery=null]){[Name=DB,Kind="Database"]}[Data], "SELECT * FROM PUBLIC.STG_CUSTOMERS", null, [EnableFolding=true])
in
Source"""

Screenshots or steps to reproduce

Expected behavior
A clear and concise description of what you expected to happen.

Version:

  • OS: [e.g. iOS]
  • Python version:
  • OpenMetadata version: [e.g. 0.8]
  • OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions