From 4c82871db17dfe5738a113355918c3d00c44b136 Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Fri, 11 Aug 2023 12:50:11 -0500 Subject: [PATCH 1/8] docs: ADR for pagination and repr of single taxonomy api --- .../0014-single-taxonomy-view-api.rst | 126 ++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 docs/decisions/0014-single-taxonomy-view-api.rst diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst new file mode 100644 index 000000000..42034fccd --- /dev/null +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -0,0 +1,126 @@ +13. Single taxonomy view API +===================================== + + +Context +-------- + +This view returns tags of a closed taxonomy (for MVP has not been implemented yet +for open taxonomies). It is necessary to make a decision about what structure the tags are going +to have and how the pagination is going to work. It was taken into account that a taxonomy can +have a large number of tags and are mostly represented as trees. + +**Branch:** Is the representation of a root Tag and all its children up to the leaves. + + +Decision +--------- + + +Tag representation +~~~~~~~~ + +Return a list of root tags and within each one have the list of children tags. Each root would have +its entire branch. The list of root tags will be ordered alphabetically as is each listing at each level of the tree. + +{ + "count": 100, + "tags": [ + { + "id": "tag_1", + "value": "Tag 1", + "taxonomy_id": "1", + "sub_tags": [ + { + "id": "tag_2", + "value": "Tag 2", + "taxonomy_id": "1", + "sub_tags": [ + (....) + ] + }, + (....) + ] + }, + (....) + ] +} + + +**Pros:** + +- The edX's interfaces show the tags in the form of a tree. +- The frontend needs no further processing as it is in a displayable format. +- It is kept as a simple implementation. + +**Cons:** + +- More implementation on the API side. + + +Pagination +~~~~~~~~~~~ + +Apply the pagination only in the root tags and bring the entire branch of each root. +Children do not affect pagination in any way. + +**Pros** + +- It is the simplest way. + +**Cons** + +- The children would not have pagination, in the long run there may be cases in which + the branch has hundreds of children, and they would still all be brought. + + +Rejected Options +----------------- + + +Render as a simple list of tags +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Return a simple list of tags, regardless of whether it is root or leaf. + +**Pros:** + +- It is simple and does not need further implementation and processing in the API. + +**Cons:** + +- It is more work to re-process all that list in the frontend to know who it is +whose father. +- In no edX's interface is it used this way and it would be a very specific use case. +- Pagination would be more complicated to perform. + + + +Get the branch in another call +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +Get the root tags in one call and all children tags of a branch in another call. +This second function is called when the user expands the parent tag. + +**Cons:** + +- In the UI there is the functionality *Expand all*, another view would have to + be made to handle this functionality in a scalable way. +- A user could make many calls; every time a parent is opened. + + + +Add the children to the pagination +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ex. If the ``page_size`` is 100, when fetching the first root tag, which has 10 children tags, +11 tags are counted for the total and there would be reamin 89 tags to be obtained. + +**Cons:** + +- If there is a branch with a number of tags that exceeds ``page_size``, + it would only return that branch. +- All branches are variable in size, therefore a variable number of root tags + would be returned. This would cause interfaces between taxonomies to be inconsistent + in the number of root tags shown. From 035e91e39e228271cf0ab0cdf7620a71ca055d0f Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Tue, 15 Aug 2023 14:46:45 -0500 Subject: [PATCH 2/8] docs: Added use cases --- docs/decisions/0014-single-taxonomy-view-api.rst | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index 42034fccd..d7044c51d 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -12,6 +12,13 @@ have a large number of tags and are mostly represented as trees. **Branch:** Is the representation of a root Tag and all its children up to the leaves. +For the decisions, the following use cases were taken into account: + +- List of root tags that can be expanded to show children tags. + - This list can be sorted alphabetically: A-Z (default) and Z-A +- The user can expand all root tags. +- The user can search for tags. + Decision --------- @@ -21,7 +28,8 @@ Tag representation ~~~~~~~~ Return a list of root tags and within each one have the list of children tags. Each root would have -its entire branch. The list of root tags will be ordered alphabetically as is each listing at each level of the tree. +its entire branch. The list of root tags will be ordered alphabetically as is each listing +at each level of the tree. This order can only be reversed in the root tag listing. { "count": 100, @@ -89,8 +97,7 @@ Return a simple list of tags, regardless of whether it is root or leaf. **Cons:** -- It is more work to re-process all that list in the frontend to know who it is -whose father. +- It is more work to re-process all that list in the frontend to know who it is whose father. - In no edX's interface is it used this way and it would be a very specific use case. - Pagination would be more complicated to perform. From ebf505ff70e115023390648286f560c3ef580ac3 Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Tue, 15 Aug 2023 15:01:23 -0500 Subject: [PATCH 3/8] docs: Added search tags decision --- .../0014-single-taxonomy-view-api.rst | 27 +++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index d7044c51d..ece57ba76 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -7,8 +7,8 @@ Context This view returns tags of a closed taxonomy (for MVP has not been implemented yet for open taxonomies). It is necessary to make a decision about what structure the tags are going -to have and how the pagination is going to work. It was taken into account that a taxonomy can -have a large number of tags and are mostly represented as trees. +to have, how the pagination is going to work and how will the search for tags be implemented. +It was taken into account that a taxonomy can have a large number of tags and are mostly represented as trees. **Branch:** Is the representation of a root Tag and all its children up to the leaves. @@ -82,6 +82,17 @@ Children do not affect pagination in any way. the branch has hundreds of children, and they would still all be brought. +Search tags +~~~~~~~~~~~~ + +Support tag search on the backend. Return a subset of matching tags in the format proposed +in this document. + +**Pros** + +- It is the most scalable way. + + Rejected Options ----------------- @@ -131,3 +142,15 @@ Ex. If the ``page_size`` is 100, when fetching the first root tag, which has 10 - All branches are variable in size, therefore a variable number of root tags would be returned. This would cause interfaces between taxonomies to be inconsistent in the number of root tags shown. + + +Search on frontend +~~~~~~~~~~~~~~~~~~ + +We constrain the number of tags allowed in a taxonomy for MVP, so that the API +can return all the tags in one page. So we can perform the tag search on the frontend. + +**Cons:** + +- It is not scalable +- Sets limits of tags that can be created in the taxonomy From d62f0530d1b29172980354e535bc9f959569493e Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Wed, 23 Aug 2023 14:07:27 -0500 Subject: [PATCH 4/8] docs: Use cases updated --- .../0014-single-taxonomy-view-api.rst | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index ece57ba76..496c88baf 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -14,10 +14,21 @@ It was taken into account that a taxonomy can have a large number of tags and ar For the decisions, the following use cases were taken into account: -- List of root tags that can be expanded to show children tags. - - This list can be sorted alphabetically: A-Z (default) and Z-A -- The user can expand all root tags. -- The user can search for tags. +- As a taxonomy administrator, I want to see all the tags available for use with a closed taxonomy, + so that I can see the taxonomy's structure in the management interface. + - As a taxonomy administrator, I want to see the available tags as a lits of root tags + that can be expanded to show children tags. + - As a taxonomy administrator, I want to sort the list of root tags alphabetically: A-Z (default) and Z-A. + - As a taxonomy administrator, I want to expand all root tags to see all children tags. + - As a taxonomy administrator, I want to search for tags, so I can find root and children tags more easily. +- (TODO: discuss with UX) As a course author, when I am editing the tags of a component, I want to see all the tags available + from a particular taxonomy that I can use. + +Excluded use cases: + +- As a content author, when searching/filtering a course/library, I want to see which tags are applied to the content + and use them to refine my search. - This is excluded from this API's use case because this is automatically handled + by elasticsearch/opensearch. Decision From b3fe84246514abb4c05cf17c061cc97d847045fa Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Fri, 25 Aug 2023 19:29:59 -0500 Subject: [PATCH 5/8] docs: Update decision to get children in another view --- .../0014-single-taxonomy-view-api.rst | 124 ++++++++++-------- 1 file changed, 69 insertions(+), 55 deletions(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index 496c88baf..9789d7f52 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -1,28 +1,32 @@ 13. Single taxonomy view API ===================================== - Context -------- This view returns tags of a closed taxonomy (for MVP has not been implemented yet for open taxonomies). It is necessary to make a decision about what structure the tags are going to have, how the pagination is going to work and how will the search for tags be implemented. -It was taken into account that a taxonomy can have a large number of tags and are mostly represented as trees. +It was taken into account that taxonomies commonly have the following characteristics: -**Branch:** Is the representation of a root Tag and all its children up to the leaves. +- It have few root tags. +- It have a very large number of children for each tag. +- It is mostly represented as trees on frontend. For the decisions, the following use cases were taken into account: - As a taxonomy administrator, I want to see all the tags available for use with a closed taxonomy, so that I can see the taxonomy's structure in the management interface. - - As a taxonomy administrator, I want to see the available tags as a lits of root tags - that can be expanded to show children tags. - - As a taxonomy administrator, I want to sort the list of root tags alphabetically: A-Z (default) and Z-A. - - As a taxonomy administrator, I want to expand all root tags to see all children tags. - - As a taxonomy administrator, I want to search for tags, so I can find root and children tags more easily. -- (TODO: discuss with UX) As a course author, when I am editing the tags of a component, I want to see all the tags available + - As a taxonomy administrator, I want to see the available tags as a lits of root tags + that can be expanded to show children tags. + - As a taxonomy administrator, I want to sort the list of root tags alphabetically: A-Z (default) and Z-A. + - As a taxonomy administrator, I want to expand all root tags to see all children tags. + - As a taxonomy administrator, I want to search for tags, so I can find root and children tags more easily. +- As a course author, when I am editing the tags of a component, I want to see all the tags available from a particular taxonomy that I can use. + - As a course author, I want to see the available tags as a lits of root tags + that can be expanded to show children tags. + - As a course author, I want to search for tags, so I can find root and children tags more easily. Excluded use cases: @@ -34,15 +38,48 @@ Excluded use cases: Decision --------- +Views & Pagination +~~~~~~~~~~~~~~~~~~~ + +Make two different views: + +- **get_root_tags():** It is the first call to obtain the parent tags of the taxonomy. + It has pagination. In addition to the common pagination metadata, it is necessary to return: + - Total number of pages. + - Total number of root tags. + - Range index of current page, Ex. Page 1: 1-12, Page 2: 13-24 + - Total number of children of each root tag. +- **get_children_tags():** Called each time the user expands a parent tag to see its children. + It has pagination. In addition to the common pagination metadata, it is necessary to return: + - Total number of children of each tag. + +As described above, the pagination of root tags and child tags are independent. +In order to be able to fulfill the functionality of "Expand-all" in a scalable way, +the following has been agreed: + +- Create a ``TAGS_THRESHOLD`` (default: 1000). +- If ``taxonomy.tags.count < TAGS_THRESHOLD``, then ``get_root_tags`` will return all tags on the taxonomy, + roots and children. +- Otherwise, ``get_root_tags`` will only return root tags, and it will be necessary + to use ``get_children_tags`` to return children. Also the "Expand-all" functionality will be disabled. + +**Pros** + +- It is the simplest way. +- Paging both root tags and children mitigates the huge number of tags that can be. + Tag representation -~~~~~~~~ +~~~~~~~~~~~~~~~~~~~ + +Return a list of root tags and within a link to obtain the children tags +or the complete list of children tags depending of ``TAGS_THRESHOLD`` (see Views & Pagination). +The list of root tags will be ordered alphabetically. If it has child tags, they must also +be ordered alphabetically. -Return a list of root tags and within each one have the list of children tags. Each root would have -its entire branch. The list of root tags will be ordered alphabetically as is each listing -at each level of the tree. This order can only be reversed in the root tag listing. +**(taxonomy.tags.count < TAGS_THRESHOLD)**:: -{ + { "count": 100, "tags": [ { @@ -60,10 +97,23 @@ at each level of the tree. This order can only be reversed in the root tag listi }, (....) ] + } + + +**Otherwise**:: + + { + "count": 100, + "tags": [ + { + "id": "tag_1", + "value": "Tag 1", + "taxonomy_id": "1", + "sub_tags_link": "http//api-call-to-get-children.com" }, (....) ] -} + } **Pros:** @@ -72,26 +122,6 @@ at each level of the tree. This order can only be reversed in the root tag listi - The frontend needs no further processing as it is in a displayable format. - It is kept as a simple implementation. -**Cons:** - -- More implementation on the API side. - - -Pagination -~~~~~~~~~~~ - -Apply the pagination only in the root tags and bring the entire branch of each root. -Children do not affect pagination in any way. - -**Pros** - -- It is the simplest way. - -**Cons** - -- The children would not have pagination, in the long run there may be cases in which - the branch has hundreds of children, and they would still all be brought. - Search tags ~~~~~~~~~~~~ @@ -124,24 +154,8 @@ Return a simple list of tags, regardless of whether it is root or leaf. - Pagination would be more complicated to perform. - -Get the branch in another call -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - -Get the root tags in one call and all children tags of a branch in another call. -This second function is called when the user expands the parent tag. - -**Cons:** - -- In the UI there is the functionality *Expand all*, another view would have to - be made to handle this functionality in a scalable way. -- A user could make many calls; every time a parent is opened. - - - -Add the children to the pagination -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Add the children to the root pagination +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Ex. If the ``page_size`` is 100, when fetching the first root tag, which has 10 children tags, 11 tags are counted for the total and there would be reamin 89 tags to be obtained. @@ -163,5 +177,5 @@ can return all the tags in one page. So we can perform the tag search on the fro **Cons:** -- It is not scalable -- Sets limits of tags that can be created in the taxonomy +- It is not scalable. +- Sets limits of tags that can be created in the taxonomy. From 940f8f86f2e2dd7d2a6d7b09a9c80d628f4ab12f Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Fri, 25 Aug 2023 20:13:47 -0500 Subject: [PATCH 6/8] docs: Add paginated search --- .../0014-single-taxonomy-view-api.rst | 30 +++++++++++-------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index 9789d7f52..9e1824dce 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -68,16 +68,31 @@ the following has been agreed: - It is the simplest way. - Paging both root tags and children mitigates the huge number of tags that can be. +Search tags +~~~~~~~~~~~~ + +Support tag search on the backend. Return a subset of matching tags. +We will use the same views to perform a search with the same logic: + +- **get_root_tags(search: str)** +- **get_children_tags(search: str)** + +For the search ``SEARCH_TAGS_THRESHOLD`` will be used. (It is recommended that it be 20 percent of ``TAGS_THRESHOLD``). +It will work in the same way of ``TAGS_THRESHOLD`` (see Views & Pagination) + +**Pros** + +- It is the most scalable way. Tag representation ~~~~~~~~~~~~~~~~~~~ Return a list of root tags and within a link to obtain the children tags -or the complete list of children tags depending of ``TAGS_THRESHOLD`` (see Views & Pagination). +or the complete list of children tags depending of ``TAGS_THRESHOLD`` or ``SEARCH_TAGS_THRESHOLD``. The list of root tags will be ordered alphabetically. If it has child tags, they must also be ordered alphabetically. -**(taxonomy.tags.count < TAGS_THRESHOLD)**:: +**(taxonomy.tags.count < *_THRESHOLD)**:: { "count": 100, @@ -123,17 +138,6 @@ be ordered alphabetically. - It is kept as a simple implementation. -Search tags -~~~~~~~~~~~~ - -Support tag search on the backend. Return a subset of matching tags in the format proposed -in this document. - -**Pros** - -- It is the most scalable way. - - Rejected Options ----------------- From 10a02a06febd7840f1ee3cce68ef1b32be5c7ebc Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Tue, 29 Aug 2023 14:16:02 -0500 Subject: [PATCH 7/8] docs: Nits and updates --- .../0014-single-taxonomy-view-api.rst | 54 +++++++++++-------- 1 file changed, 31 insertions(+), 23 deletions(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index 9e1824dce..64a114703 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -1,4 +1,4 @@ -13. Single taxonomy view API +14. Single taxonomy view API ===================================== Context @@ -9,9 +9,9 @@ for open taxonomies). It is necessary to make a decision about what structure th to have, how the pagination is going to work and how will the search for tags be implemented. It was taken into account that taxonomies commonly have the following characteristics: -- It have few root tags. -- It have a very large number of children for each tag. -- It is mostly represented as trees on frontend. +- It has few root tags. +- It may a very large number of children for each tag. +- It is mostly represented as trees on frontend, with a depth of up to 3 levels. For the decisions, the following use cases were taken into account: @@ -41,43 +41,51 @@ Decision Views & Pagination ~~~~~~~~~~~~~~~~~~~ -Make two different views: +Make one view: -- **get_root_tags():** It is the first call to obtain the parent tags of the taxonomy. - It has pagination. In addition to the common pagination metadata, it is necessary to return: - - Total number of pages. - - Total number of root tags. - - Range index of current page, Ex. Page 1: 1-12, Page 2: 13-24 - - Total number of children of each root tag. -- **get_children_tags():** Called each time the user expands a parent tag to see its children. - It has pagination. In addition to the common pagination metadata, it is necessary to return: - - Total number of children of each tag. +**get_matching_tags(parent_tag_id: str = None, search_term: str = None)** -As described above, the pagination of root tags and child tags are independent. +that can handle this cases: + +- Get the root tags of the taxonomy. If ``parent_tag_id`` is ``None``. +- Get the children of a tag. Called each time the user expands a parent tag to see its children. + If ``parent_tag_id`` is not ``None``. + +In both cases the results are paginated. In addition to the common pagination metadata, it is necessary to return: + +- Total number of pages. +- Total number of root/children tags. +- Range index of current page, Ex. Page 1: 1-12, Page 2: 13-24. +- Total number of children of each root tag. + +The pagination of root tags and child tags are independent. In order to be able to fulfill the functionality of "Expand-all" in a scalable way, the following has been agreed: - Create a ``TAGS_THRESHOLD`` (default: 1000). -- If ``taxonomy.tags.count < TAGS_THRESHOLD``, then ``get_root_tags`` will return all tags on the taxonomy, +- If ``taxonomy.tags.count < TAGS_THRESHOLD``, then ``get_matching_tags()`` will return all tags on the taxonomy, roots and children. -- Otherwise, ``get_root_tags`` will only return root tags, and it will be necessary - to use ``get_children_tags`` to return children. Also the "Expand-all" functionality will be disabled. +- Otherwise, ``get_matching_tags()`` will only return paginated root tags, and it will be necessary + to use ``get_matching_tags()`` to return paginated children. Also the "Expand-all" functionality will be disabled. + +For search you can see the next section (Search tags) **Pros** - It is the simplest way. -- Paging both root tags and children mitigates the huge number of tags that can be. +- Paging both root tags and children mitigates the huge number of tags that can be found in large taxonomies. Search tags ~~~~~~~~~~~~ Support tag search on the backend. Return a subset of matching tags. -We will use the same views to perform a search with the same logic: +We will use the same view to perform a search with the same logic: + +**get_matching_tags(parent_tag_id: str = None, search_term: str = None)** -- **get_root_tags(search: str)** -- **get_children_tags(search: str)** +We can use ``search_term`` to perferom a search on root tags or children tags depending of ``parent_tag_id``. -For the search ``SEARCH_TAGS_THRESHOLD`` will be used. (It is recommended that it be 20 percent of ``TAGS_THRESHOLD``). +For the search, ``SEARCH_TAGS_THRESHOLD`` will be used. (It is recommended that it be 20% of ``TAGS_THRESHOLD``). It will work in the same way of ``TAGS_THRESHOLD`` (see Views & Pagination) **Pros** From dc4b7927f78537505e6bc1346c4f542cfdb6e8a4 Mon Sep 17 00:00:00 2001 From: XnpioChV Date: Tue, 29 Aug 2023 15:13:48 -0500 Subject: [PATCH 8/8] docs: nit --- docs/decisions/0014-single-taxonomy-view-api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/decisions/0014-single-taxonomy-view-api.rst b/docs/decisions/0014-single-taxonomy-view-api.rst index 64a114703..a71a1ce7f 100644 --- a/docs/decisions/0014-single-taxonomy-view-api.rst +++ b/docs/decisions/0014-single-taxonomy-view-api.rst @@ -56,7 +56,7 @@ In both cases the results are paginated. In addition to the common pagination me - Total number of pages. - Total number of root/children tags. - Range index of current page, Ex. Page 1: 1-12, Page 2: 13-24. -- Total number of children of each root tag. +- Total number of children of each tag. The pagination of root tags and child tags are independent. In order to be able to fulfill the functionality of "Expand-all" in a scalable way,