[RFC] Filter composition

[teohhanhui]

Instead of giving the client the ability to construct arbitrary queries / filtering (something better served by existing standards / specifications, e.g. OData and GraphQL), we could allow predefined composite filters in the API, each composed of one or more filter primitives.

For example:

/users
q: match('username', 'ipartial') or match('email', 'ipartial') or match('firstName', 'iword_start') or match('lastName', 'iword_start')

/products
q: match('sku', 'iexact') or match('name', 'iword_start') or match('description', 'iword_start')

This also supports the aliasing use case where you could easily define multiple filter parameters for the same property, e.g.:

/users
email: match('email', 'iexact')
email_partial: match('email', 'ipartial')

/products
sku: match('sku', 'iexact')
sku_partial: match('sku', 'ipartial')

Of course, this also forces us to have decoupled code between the declaration of filter parameters (currently the $properties argument) and the actual filtering (currently the filterProperty function). It'd be a good opportunity to get rid of inheritance of filter classes (mark as deprecated; to be removed in API Platform 3.0).

Related:

• [RFC] Add a where filter. #1832
• WhereFilter - Allow to build complex search query #2055
• Search filter: configure at query level rather than definition level #2351
• How to have an OR filter api-platform#577
• API Platform "OR" and "IN" filters  api-platform#639

[dunglas]

There are 2 other popular syntaxes:

• Postgrest syntax: http://postgrest.org/en/v5.1/api.html
• Loopback: https://loopback.io/doc/en/lb2/Where-filter.html

Both already have a lot of compatible client tools (including React Admin providers IIRC) and would be fine to me.

[dunglas]

I actually like the loopback syntax because it looks very PHPish.

[teohhanhui]

My suggestion here has nothing to do with complex querying from the client. These composite filter parameters are defined by the API. In fact, these 2 approaches could coexist.

Also, this would make it easier to write custom filters by being able to use composition of filter primitives. Whether we want to expose these primitives through configuration is another story.

[ragboyjr]

regarding query syntax, https://github.com/krakphp/aql is a solution I built that basically allows sql type expressions in the API, but it's parsed and validated and can contain certain constraints, it might be something of use here for allowing more complex querying/flexibility

[teohhanhui]

I'm strongly -1 for giving too much control to the client (there's GraphQL for that lol, at least it's better than inventing yet another format / syntax) (or OData if we go that route).

Anyway, it's irrelevant to this RFC, which is about filter composition on the server side only.

[ragboyjr]

ah, i guess I misunderstood your original post, the idea is that if we do:

/users?q=abc123

We could define a proper query portion in the resource of what q maps to. So instead of q mapping to a simple field, we could build up a complex expression for q is searched on.

Is that understanding correct?

[teohhanhui]

@ragboyjr Yes 😄

[ragboyjr]

ah, ok, cool beans, y, this would be awesome, part of me wonders if maybe just making the custom filter API easier would fix this. Like, if making filters was just more composable already, then we could easily have filter factories that can be used to create filters instead of building our own expression parser or something on the backend.

[teohhanhui]

@ragboyjr Could you elaborate? I'm not sure I get what you mean, about the composable filter API and filter factories.

[ragboyjr]

Sure,

The basic idea would be similar to this: https://github.com/krakphp/php-inc/blob/master/src/php-inc.php#L30

Where we have filter classes that decorate certain things
We'd provide decorating filters like AllFilter and AnyFilter, which take a list of filter interfaces and will delegate to their children when apply or filterProperty is called.

Then in the filter config for a resource, we'd define structured config like:

@ApiFilter(
    ASTFilter::class,
    config={
        "type": "or",
        "filters": [
            {
                "type": "property", 
                "property": "username", 
                "filter": { "type": "match", "partial": true, "ignoreCase": true }
            },
            {
                "type": "property", 
                "property": "email", 
                "filter": { "type": "match", "partial": true, "ignoreCase": true }
            }
        ]
    }
)

The config is a bit verbose, but the idea is that we'd introduce a few new filters which handled one thing: OrFilter that takes a set of filters in its constructor. PropertyFilter, that takes on property key and the filter to apply on the property value. And a MatchFilter which is basically our search filter which takes some configuration parameters on if it's caseless and a partial match or not.

[teohhanhui]

Please see my examples in the first post. We could use expression language. No need for verbose config. But the real problem is Doctrine's QueryBuilder. It's not composable at all.

[ymarillet]

any update about this feature ? :-)

[teohhanhui]

any update about this feature ? :-)

Not unless we are able to switch to a composable query builder (on top of / as an alternative to Doctrine's QueryBuilder). Perhaps it need not (and should not) support the same expressiveness, which might be an impediment to composability indeed.