Problem
Our api needs support for pagination in order to deal with initial syncing of github requests
http://justincalleja.com/2016/01/26/elixir-streams-reload/#How_to_fetch_Github_organization_repos_and_their_pagination has a pretty nice writeup on how we could potentially implement that.
However, while streams can be done in paralel, the write up in that link does it sequentially, since it relies on links.next_page to fetch each page of results.
GitHub does not recommended this, but we definitely could
-
get the first page to figure out how many pages there are
-
create a parallel request for each page
-
entrypoint should be CodeCorps.GitHub
-
since pagination is on GET requests, there should not be a need to support specifying the method as an argument - get_all or something like that would work
-
it's used for syncing, so it should be an integration request
Once we have that, we could continue by adding higher level functions to our individual endpoints, in separate issues.
GitHub.API.Installation.repositories may have to be enhanced to support this
GitHub.API.Repository.issues needs to be implemented
GitHub.API.Issue.comments needs to be implemented
Problem
Our api needs support for pagination in order to deal with initial syncing of github requests
http://justincalleja.com/2016/01/26/elixir-streams-reload/#How_to_fetch_Github_organization_repos_and_their_pagination has a pretty nice writeup on how we could potentially implement that.
However, while streams can be done in paralel, the write up in that link does it sequentially, since it relies on
links.next_pageto fetch each page of results.GitHub does not recommended this, but we definitely could
get the first page to figure out how many pages there are
create a parallel request for each page
entrypoint should be
CodeCorps.GitHubsince pagination is on GET requests, there should not be a need to support specifying the method as an argument -
get_allor something like that would workit's used for syncing, so it should be an integration request
Once we have that, we could continue by adding higher level functions to our individual endpoints, in separate issues.
GitHub.API.Installation.repositoriesmay have to be enhanced to support thisGitHub.API.Repository.issuesneeds to be implementedGitHub.API.Issue.commentsneeds to be implemented