Support for contains function in datafusion substrait consumer

[richtia]

Is your feature request related to a problem or challenge?

Datafusion does not have a contains function and will return an exception if a substrait plan contains it

Exception: DataFusion error: NotImplemented("Unsupported function name: \"contains:str_str\"")

Describe the solution you'd like

When the datafusion consumer sees the contains function, it can convert it to a combination of strpos + >

strpos(lit("abcd"), lit("de") > 0

Describe alternatives you've considered

No response

Additional context

No response

[Lordworms]

take

[Lordworms]

Do we really need this conversion? what about adding a new ScalarFunction 'contains'? @alamb I would like to receive your advice here.

[alamb]

I am not sure @Lordworms -- maybe @waynexia or @Blizzara have more context / understanding

@richtia does this request come from a particular query you are runniing?

[richtia]

My preference would actually be to add the function as well. The request came from trying to run a query with the contains function.

[Blizzara]

+1 to adding the function into DataFusion (+ the relevant mapping into Substrait) if it doesn't exist yet.

Seems like Arrow has a contains function but I cannot find it in DataFusion so maybe it doesn't indeed exist yet, dunno if the Arrow version can somehow be easily used?

[Lordworms]

+1 to adding the function into DataFusion (+ the relevant mapping into Substrait) if it doesn't exist yet.

Seems like Arrow has a contains function but I cannot find it in DataFusion so maybe it doesn't indeed exist yet, dunno if the Arrow version can somehow be easily used?

Sure, I'll file a PR later today

[richtia]

+1 to adding the function into DataFusion (+ the relevant mapping into Substrait) if it doesn't exist yet.

Seems like Arrow has a contains function but I cannot find it in DataFusion so maybe it doesn't indeed exist yet, dunno if the Arrow version can somehow be easily used?

I didn't see the function in DataFusion either. I also couldn't find it in Arrow, at least not under any compute function.

[Lordworms]

+1 to adding the function into DataFusion (+ the relevant mapping into Substrait) if it doesn't exist yet.
Seems like Arrow has a contains function but I cannot find it in DataFusion so maybe it doesn't indeed exist yet, dunno if the Arrow version can somehow be easily used?

I didn't see the function in DataFusion either. I also couldn't find it in Arrow, at least not under any compute function.

I think DuckDB got a similar function, let me know if it is ok for me to implement similar behavior like this

[richtia]

Yeah, duckdb has a contains function as well as the substrait support. The same substrait plans that gave me the datafusion errors were able to execute against the DuckDB substrait consumer.