[SPARK-52851][SDP] Remove public APIs for append once flows

[JiaqiWang18]

What changes were proposed in this pull request?

Remove the once arg in @append_flow decorator for SDP

def append_flow(
    *,
    target: str,
    name: Optional[str] = None,
    spark_conf: Optional[Dict[str, str]] = None,
    once: bool = False,                                              # <--- removed
) -> Callable[[QueryFunction], None]:

Also removes once field from the DefineFlow proto.

The SQL API CreateFlowCommand does not take in a once argument so no change is needed there. The argument is hardcoded to be False: (source)

Why are the changes needed?

The Declarative Pipelines append_flow decorator includes a once argument, which, if True, indicates the flow should run only once. (It will be rerun upon a full refresh operation.)

However, the server does not currently implement this behavior yet. To avoid accidentally releasing APIs that don't actually work, we should take these arguments out for now. And add them back in when we actually support this functionality.

Does this PR introduce any user-facing change?

Yes, an argument was removed from a public API. However, SDP is not released yet, so there will be minimal user impact.

How was this patch tested?

Modified existing testcases to ensure consistency.

Was this patch authored or co-authored using generative AI tooling?

No

[JiaqiWang18]

@anishm-db @sryza

[sryza]

Thanks for picking this up @JiaqiWang18 . Are we also able to remove property from the DefineFlow proto definition – that's technically a public API as well. And also, is their SQL syntax that supports this?

[JiaqiWang18]

Thanks for picking this up @JiaqiWang18 . Are we also able to remove property from the DefineFlow proto definition – that's technically a public API as well. And also, is their SQL syntax that supports this?

Yeah, just removed once from DefineFlow. The SQL syntax doesn't support using ONCE in the query according to this CreateFlowCommand. It is hardcoding the value to be false here.

[sryza]

LGTM!

[sryza]

Merged to master