feat(recent-files): allow grouping search results by mime type

[cristianscheid]

• Resolves: #

Summary

Introduces file grouping logic on the backend for the SEARCH remote.php/dav endpoint. When enabled, files of certain MIME types (configurable) that were uploaded/created/modified close together in time are returned with a <nc:mime_type_group> prop containing the group they belong to.

Behavior

When group_recent_files is enabled and <nc:mime_type_group> is requested, grouping is applied to the results. Each group is represented by an integer and returned inside the <nc:mime_type_group> prop.

• If an element belongs to group 1:

<d:response>
    <d:href>/remote.php/dav/files/admin/mock_image.jpg</d:href>
    <d:propstat>
        <d:prop>
            <d:displayname>mock_image.jpg</d:displayname>
            <d:getcontenttype>image/jpeg</d:getcontenttype>
            <nc:last_activity>1783729020</nc:last_activity>
            <nc:mime_type_group>1</nc:mime_type_group> // element belongs to group 1
        </d:prop>
        <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
</d:response>

• If an element does not belong to a group, <nc:mime_type_group> is returned as 404:

<d:response>
    <d:href>/remote.php/dav/files/admin/other_mock_image.jpg</d:href>
    <d:propstat>
        <d:prop>
            <d:displayname>other_mock_image.jpg</d:displayname>
            <d:getcontenttype>image/jpeg</d:getcontenttype>
            <nc:last_activity>1783728840</nc:last_activity>
        </d:prop>
        <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
        <d:prop>
            <nc:mime_type_group/>
        </d:prop>
        <d:status>HTTP/1.1 404 Not Found</d:status> // element does not belong to a group
    </d:propstat>
</d:response>

See the test scenario below for more details.

Test scenario

Configs were set like below:

occ config:app:set files group_recent_files --value=true
occ config:app:set files recent_files_group_mime_types --value='["image/avif","image/gif","image/heic","image/heif","image/jpeg","image/jpg","image/jxl","image/png","image/tiff","image/webp"]'
occ config:app:set files recent_files_group_same_folder_only --value=true
occ config:app:set files recent_files_group_min_group_size --value=2
occ config:app:set files recent_files_group_timespan_minutes --value=2
occ config:app:set files recent_files_group_collapsed_items_limit --value=25

• group_recent_files - enables/disables file grouping
  • defaults to false
• recent_files_group_mime_types - MIME types eligible for grouping
  • defaults to []
• recent_files_group_same_folder_only - whether files must be in the same folder to be grouped together
  • defaults to true
• recent_files_group_min_group_size - minimum number of files needed to form a group
  • defaults to 2
• recent_files_group_timespan_minutes - maximum time gap between consecutive items for them to be grouped together
  • defaults to 2
• recent_files_group_collapsed_items_limit - maximum number of collapsed items the response should contain; a "collapsed item" considers each group as a single item, regardless of how many files it contains
  • defaults to 25

With the config above, files were uploaded as follows:

• img_1.jpg and img_2.jpg at minute 6
• json_3.json and img_4.jpg at minutes 9 and 10, respectively
• img_5.jpg and img_6.jpg at minute 14
• img_7.jpg and img_8.jpg at minute 17

The time used for sorting/grouping is <nc:last_activity>, which represents max($uploadTime, $creationTime, $lastModified). In this scenario, upload time is the max for all files.

For grouping to work correctly, the request must order by <nc:last_activity> descending:

<d:orderby>
    <d:order>
        <d:prop>
            <nc:last_activity/>
        </d:prop>
        <d:descending/>
    </d:order>
</d:orderby>

This returns results from most to least recent, meaning each item's <nc:last_activity> is greater than or equal to the next item's.

Grouping logic

For each item, in order:

• if it's non-groupable (wrong mimetype), it gets no group
• if it's groupable, look ahead at the following items within the time window (defined by recent_files_group_timespan_minutes):
  • if any of the following items in the time window is non-groupable, the window is "contaminated" and no group is assigned
  • if recent_files_group_same_folder_only is enabled and any of the following items in the time window is from a different folder, the window is also "contaminated" and no group is assigned
• if the window isn't contaminated, but the resulting group has fewer items than recent_files_group_min_group_size, no group is assigned either
• otherwise, the same group number is assigned to all items in the window

Extra fetches

Each fetch is capped by the request's <d:limit> (defaulting to 100 if not set). If a single fetch doesn't return enough collapsed items to reach recent_files_group_collapsed_items_limit, additional fetches are made until it does or until there are no more items to fetch (a limit of 5 extra fetches was set as a safety cap).

Using the scenario above (recent_files_group_collapsed_items_limit = 25), requesting with:

<d:limit>
    <d:nresults>5</d:nresults>
</d:limit>

returns 5 raw items on the first fetch, but fewer than 25 collapsed items, since img_7.jpg/img_8.jpg and img_5.jpg/img_6.jpg each count as one group. So another fetch is made to attempt to reach the configured value.

Additionally, if a page ends on a groupable node, one more fetch is made to check whether the group continues on the next page. This avoids splitting a group across two pages.

Test result

Given the scenario above, results are returned and grouped as:

Files	Group
img_7.jpg, img_8.jpg	1
img_5.jpg, img_6.jpg	2
img_4.jpg	none - next item in the window (json_3.json) is non-groupable
json_3.json	none - non-groupable
img_1.jpg, img_2.jpg	3

Result of request/response can be seen below:

// Request
curl -X SEARCH "http://nextcloud.local/remote.php/dav/" \
  -u "admin:admin" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<d:searchrequest xmlns:d="DAV:" xmlns:nc="http://nextcloud.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:ocs="http://open-collaboration-services.org/ns" xmlns:ns="https://github.com/icewind1991/SearchDAV/ns">
    <d:basicsearch>
        <d:select>
            <d:prop>
                <d:displayname />
                <d:getcontenttype />
                <nc:last_activity />
                <nc:mime_type_group />
            </d:prop>
        </d:select>
        <d:from>
            <d:scope>
                <d:href>/files/admin/</d:href>
                <d:depth>infinity</d:depth>
            </d:scope>
        </d:from>
        <d:where>
            <d:not>
                <d:eq>
                    <d:prop>
                        <d:getcontenttype/>
                    </d:prop>
                    <d:literal>httpd/unix-directory</d:literal>
                </d:eq>
            </d:not>
        </d:where>
        <d:orderby>
            <d:order>
                <d:prop>
                    <nc:last_activity/>
                </d:prop>
                <d:descending/>
            </d:order>
        </d:orderby>
        <d:limit>
            <d:nresults>5</d:nresults>
        </d:limit>
    </d:basicsearch>
</d:searchrequest>'

// Response
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_7.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_7.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783729020</nc:last_activity>
                <nc:mime_type_group>1</nc:mime_type_group>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_8.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_8.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783729020</nc:last_activity>
                <nc:mime_type_group>1</nc:mime_type_group>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_5.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_5.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783728840</nc:last_activity>
                <nc:mime_type_group>2</nc:mime_type_group>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_6.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_6.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783728840</nc:last_activity>
                <nc:mime_type_group>2</nc:mime_type_group>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_4.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_4.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783728600</nc:last_activity>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
        <d:propstat>
            <d:prop>
                <nc:mime_type_group/>
            </d:prop>
            <d:status>HTTP/1.1 404 Not Found</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/json_3.json</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>json_3.json</d:displayname>
                <d:getcontenttype>text/plain</d:getcontenttype>
                <nc:last_activity>1783728540</nc:last_activity>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
        <d:propstat>
            <d:prop>
                <nc:mime_type_group/>
            </d:prop>
            <d:status>HTTP/1.1 404 Not Found</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_2.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_2.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783728360</nc:last_activity>
                <nc:mime_type_group>3</nc:mime_type_group>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
    <d:response>
        <d:href>/remote.php/dav/files/admin/img_1.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:displayname>img_1.jpg</d:displayname>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
                <nc:last_activity>1783728360</nc:last_activity>
                <nc:mime_type_group>3</nc:mime_type_group>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
</d:multistatus>

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)
• Labels added where applicable (ex: bug/enhancement, 3. to review, feature component)
• Milestone added for target branch/version (ex: 32.x for stable32)

AI (if applicable)

• The content of this PR was partly or fully generated using AI