Paging

[matt-ramotar]

Context:

• Closes [BUG] State Flow From launchPagingStore Does Not Reflect Latest Writes in Mutable Store #602
• Addresses @yigit feedback Fix launchPagingStore #603 (comment)
• Addresses Best approach for Pager and pagination #608
• Separates Paging #611 into smaller PRs
• Closes Improve StoreKey API #604

Description

Introducing a solution for paging in KMP projects. Motivations:

1. Seamless integration with Store and Mutable Store
2. Support for local mutations and streaming of child items within the list of paging items
3. Support for custom reducer, middleware, and post-reducer effects

Type of Change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Test Plan

Unit tests

Checklist:

Before submitting your PR, please review and check all of the following:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my change is effective
• New and existing unit tests pass locally with my changes

Additional Notes:

Paging Technical Design Doc

1. Overview

Modular and flexible architecture. Using builder, reducer, middleware, and post-reducer effect patterns. Unidirectional
data flow. The Pager is the main component. Actions are dispatched through the Pager. The PagerBuilder creates
the Pager. It allows configuration of the paging behavior. The PagingSource defines the data loading logic. The FetchingStrategy determines when to fetch the next page. The AggregatingStrategy combines loaded pages into a single list. The Reducer handles state changes based on actions. When an action is dispatched, it goes through the middleware pipeline. The middleware can modify the action. The reducer then updates the state based on the action. After the reducer, we invoke post-reducer effects associated with the action and new state. The updated state is sent back the Pager and emitted to the UI.

2. Key Components

• Pager: The main entry point for the paging library. It coordinates the paging process and provides access to the paging state and data.
• PagingState: Represents the current state of the paging data, including loaded pages, errors, and loading status.
• PagingAction: Defines the actions that can be dispatched to modify the paging state.
• Reducer: Responsible for taking the current paging state and a dispatched action, and producing a new paging state based on the action and the current state.
• Middleware: Intercepts and modifies paging actions before they reach the reducer, allowing for pre-processing, logging, or any other custom logic.
• Effect: Represents side effects or additional actions that need to be performed after the state has been reduced based on a dispatched action.
• PagingSource: Represents a data source that provides paged data, emitting a stream of load results.
• PagingBuffer: A custom data structure for efficiently storing and retrieving paging data.

3. Customizations

Providing many extension points and customization options to tailor behavior. Main customization points:

• PagingConfig: Allows configuring the paging behavior, such as page size, prefetch distance, and insertion strategy.
• FetchingStrategy: Determines whether to fetch more data based on the current state of the pager.
• AggregatingStrategy: Defines how loaded pages of data should be combined and ordered to form a coherent list of paging items.
• ErrorHandlingStrategy: Specifies different strategies for handling errors during the paging process.
• UserCustomActionReducer: Allows defining custom reducers for handling user-defined actions.

4. Data Flow

Unidirectional data flow. Main steps:

1. Pager is configured using PagerBuilder and provided an initial key, flow of anchor position, and paging config.
2. Pager subscribes to the PagingSource to receive paging data updates.
3. When a PagingAction is dispatched, it goes through the configured Middleware chain. This enables interception and modification of the action.
4. The modified action reaches the Reducer, which reduces the current PagingState based on the action and returns a new PagingState.
5. After reduction, any configured Effect instances are launched, enabling side effects to be performed based on the new PagingState.
6. Pager updates StateManager with the new PagingState.
7. FetchingStrategy determines when to fetch the next page of data based on the PagingConfig and current PagingState.
8. When a new page needs to be fetched, QueueManager enqueues the page key, and the JobCoordinator coordinates the execution of the paging job.
9. PagingSource loads the requested page and emits the loaded data through the PagingSourceStreamProvider.
10. The loaded page is stored in the MutablePagingBuffer for efficient retrieval and aggregation.
11. The AggregatingStrategy aggregates the loaded pages into a single list, which is then emitted through the Pager for consumption by the UI.

5. Sample Code

See https://github.com/MobileNativeFoundation/Store/tree/paging/paging

[digitalbuddha]

is cache not needed anymore?

[digitalbuddha]

you can also do org.mobilenativefoundation.store5.paging.core

in case you want to ever have something like 5/6 at same time

[digitalbuddha]

I still prefer Key,Request,Response as the param names :-)

[OliverRhyme]

Any updates on this?

[matt-ramotar]

@OliverRhyme sorry to be slow. Update here: #671 (comment)

[tyvsmith]

A few high level comments:

1. Are there any pagination standards that should be implementing or following by default, ie IETF cursor based pagination?
2. Do you need to supply any specific test utils or fakes for the end-user to idiomatically test their pagination impls?
3. There is limited use of internal or private visibility, even in impl package. Should everything be visible to the consumer?