Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
4947852
Use alternative library for passport-github
May 5, 2021
d4f3e97
Updating with new octokit endpoints
ardelato May 5, 2021
5dca5e7
Modified the last portions where we used the old endpoints
ardelato May 6, 2021
59a69c4
Add missing spacing
addison-grant Jun 9, 2021
906349c
Git-ignore DB configuration settings, .env.db
Jun 9, 2021
5ffc6a3
Age: Match preexisting colors
mlahargou May 4, 2021
fbc7679
Bump url-parse from 1.4.7 to 1.5.1
dependabot[bot] May 10, 2021
4f4208c
Bump hosted-git-info from 2.8.5 to 2.8.9
dependabot[bot] May 9, 2021
4ba49d1
Bump underscore from 1.9.1 to 1.12.1
dependabot[bot] May 6, 2021
d193d5f
Bump lodash from 4.17.19 to 4.17.21
dependabot[bot] May 6, 2021
3edc303
Add reviews table to store pull_request_reviews info
davidrans Jun 2, 2021
3c37e63
Add model objects for new table
davidrans Jun 2, 2021
ac6dc90
db-manager: add methods for getting/setting reviews
davidrans Jun 2, 2021
8d62ad6
Insert/update reviews table when updating pull
davidrans Jun 2, 2021
3f17d8c
Remove outdated comment
davidrans Jun 2, 2021
ec6d5c8
De-duplicate signature updating logic
davidrans Jun 2, 2021
e9dc37e
De-duplicate signature parsing logic
davidrans Jun 2, 2021
9767fcd
Pull Request Review: handle webhooks
davidrans Jun 2, 2021
e1a1dc9
Bump dns-packet from 1.3.1 to 1.3.4
dependabot[bot] May 29, 2021
4f099cb
Update octokit/rest call to newer form
Jun 9, 2021
ae56cb1
Increase retry limit
Jun 9, 2021
46b1025
Make .then chain more readable by separating
Jun 9, 2021
701adfc
Paginate results for getAllIssues
Jun 9, 2021
bf12a09
Add newline at end of file.
Jun 9, 2021
b69f41b
No need to paginate single-item result
Jun 9, 2021
35e5132
Update calls to get PR reviews
Jun 9, 2021
3394d44
Delete rate-limit.js
addison-grant Jun 14, 2021
92d5118
Remove reference to rate-limit.js
addison-grant Jun 15, 2021
96ff0af
Merge remote-tracking branch 'origin/master' into update-github-passp…
Jun 15, 2021
0a2f0d3
Merge remote-tracking branch 'origin/master' into update-github-passp…
Jul 13, 2021
97b09cc
Update reviews.body to be nullable
Jul 13, 2021
381b53c
Update schema.sql file with latest migration
Jul 13, 2021
0476e3b
Authentication: update team membership api call
danielbeardsley Aug 7, 2021
ba47d4c
git-manager: extract data attribute
danielbeardsley Aug 7, 2021
f74bae4
git-manager: fix debug log, add repo
danielbeardsley Aug 7, 2021
7a65e0c
git-manager: fix getIssueEvents call
danielbeardsley Aug 7, 2021
aa9d0ff
git-manager: remove duplicate function
danielbeardsley Aug 10, 2021
21c469a
promise.done: don't use prototype function
danielbeardsley Aug 10, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ node_modules
/views/standard/css
pulldasher.pid
dist
/.env.db
2 changes: 1 addition & 1 deletion config.example.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ module.exports = {
hook_secret: "some random string to use (?secret=oxwm5gks) to 'secure' github hook handlers",
// Limit access to specific users or teams.
requireOrg: "Limit access to users belonging to this github organization",
requireTeam: "[Optional] Limit access to users belonging to this team name within the above github organization"
requireTeam: "[Optional] Limit access to users belonging to this team name within the above github organization, uses the slug version of the team name, i.e. 'some-team' from @SomeOrg/some-team"
},
session: {
secret: "secret for signing session cookies"
Expand Down
23 changes: 4 additions & 19 deletions lib/authentication.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ var config = require('./config-loader'),
github = require('./git-manager').github,
passport = require('passport'),
_ = require('underscore'),
GitHubStrategy = require('passport-github').Strategy;

@deltuh-vee deltuh-vee Jul 16, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test dev_block 👍

hello -Addison

GitHubStrategy = require('passport-github2').Strategy;
Comment thread
addison-grant marked this conversation as resolved.

const FAKE_USER = process.env.MOCK_AUTH_AS_USER;

Expand Down Expand Up @@ -98,27 +98,12 @@ module.exports = {
let checkOrgMembershipRequirements;

if (config.github.requireTeam) {
// All this logic is needed to get the team id from a team name
const getTeamId = github.orgs.getTeams({org: config.github.requireOrg})
.then(function(teams) {
const team = _.findWhere(teams, {name: config.github.requireTeam});
if (!team) {
const m = "Team " + (config.github.requireTeam) + " not found in Organization: " + config.github.requireOrg;
debug(m);
console.error(m); // eslint-disable-line no-console
process.exit(1);
}
return team.id;
});

checkOrgMembershipRequirements = function(options) {
return getTeamId.then(function(teamId) {
options.id = teamId;
return github.orgs.getTeamMembership(options);
});
options.team_slug = config.github.requireTeam;
return github.teams.getMembershipForUserInOrg(options);
};
} else {
checkOrgMembershipRequirements = github.orgs.checkMembership;
checkOrgMembershipRequirements = github.orgs.checkMembershipForUser;
}

function confirmOrgMembership(user) {
Expand Down
106 changes: 52 additions & 54 deletions lib/git-manager.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
var GithubApi = require('@octokit/rest'),
var { Octokit } = require('@octokit/rest'),
{ throttling } = require("@octokit/plugin-throttling"),
MyOctokit = Octokit.plugin(throttling),
config = require('./config-loader'),
Promise = require('bluebird'),
_ = require('underscore'),
github = new GithubApi({
version: '3.0.0'
}),
debug = require('./debug')('pulldasher:github'),
utils = require('./utils'),
Pull = require('../models/pull'),
Expand All @@ -14,25 +13,45 @@ var GithubApi = require('@octokit/rest'),
Label = require('../models/label'),
Status = require('../models/status'),
Signature = require('../models/signature'),
getLogin = require('./get-user-login'),
rateLimit = require('./rate-limit.js');
getLogin = require('./get-user-login');

const github = new MyOctokit({
auth: config.github.token,
throttle: {
onRateLimit: (retryAfter, options) => {
github.log.warn(
`Request quota exhausted for request ${options.method} ${options.url}`
);

// Retry five times after hitting a rate limit error, then give up
if (options.request.retryCount <= 5) {
github.log.debug(`Retrying after ${retryAfter} seconds!`);
return true;
}
},
onAbuseLimit: (retryAfter, options) => {
// does not retry, only logs a warning
github.log.warn(
`Abuse detected for request ${options.method} ${options.url}`
);
},
}
});

const githubRest = github.rest;

github.authenticate({
type: 'oauth',
token: config.github.token
});

module.exports = {
github: github,
github: githubRest,

/**
* Returns a promise which resolves to a GitHub API response to
* a query for a particular Pull Request.
*/
getPull: function(repo, number) {
debug("Getting pull %s", number);
return rateLimit(github.pullRequests.get)(params({ number }, repo))
.then(res => res.data);
return githubRest.pulls.get(params({ pull_number: number }, repo))
.then(res => res.data);
},

/**
Expand All @@ -42,7 +61,7 @@ module.exports = {
*/
getOpenPulls: function(repo) {
debug("Getting open pulls in repo %s", repo);
return rateLimit(github.pullRequests.getAll)(params({}, repo)).then(paginate);
return github.paginate(githubRest.pulls.list, params({state: 'open'}, repo));
},

/**
Expand All @@ -52,8 +71,7 @@ module.exports = {
*/
getAllPulls: function(repo) {
debug("Getting all pulls in repo %s", repo);
return rateLimit(github.pullRequests.getAll)(params({ state: 'all' }, repo))
.then(paginate);
return github.paginate(githubRest.pulls.list, params({ state: 'all'}, repo));
},

/**
Expand All @@ -62,11 +80,11 @@ module.exports = {
* Returns a promise which resolves to a github issue
*/
getIssue: function(repo, number) {
const searchParams = params({ number }, repo);
const searchParams = params({ issue_number: number }, repo);
debug("Getting issue %s in repo %s", number, repo);
return rateLimit(github.issues.get)(searchParams)
.then(res => res.data)
.then(addRepo(searchParams));
return githubRest.issues.get(searchParams)
.then(res => res.data)
.then(addRepo(searchParams));
},

/**
Expand All @@ -75,12 +93,11 @@ module.exports = {
* Returns a promise which resolves to an array of all open issues
*/
getOpenIssues: function(repo) {
const searchParams = params({}, repo);
debug("Getting open issues");
return rateLimit(github.issues.getForRepo)(searchParams)
.then(paginate)
.then(filterOutPulls)
.then(addRepo(searchParams));
const searchParams = params({state: 'open'}, repo);
debug("Getting open issues in repo %s", repo);
return github.paginate(githubRest.issues.listForRepo, searchParams)
.then(filterOutPulls)
.then(addRepo(searchParams));
},

/**
Expand All @@ -91,10 +108,9 @@ module.exports = {
getAllIssues: function(repo) {
const searchParams = params({state: 'all'}, repo);
debug("Getting all issues");
return rateLimit(github.issues.getForRepo)(searchParams)
.then(paginate)
.then(filterOutPulls)
.then(addRepo(searchParams));
return github.paginate(githubRest.issues.listForRepo, searchParams)
.then(filterOutPulls)
.then(addRepo(searchParams));
},

/**
Expand Down Expand Up @@ -225,18 +241,6 @@ module.exports = {
},
};

async function paginate(res) {
let response = res;
let all = res.data;

while (github.hasNextPage(response)) {
response = await rateLimit(github.getNextPage)(response);
all = all.concat(response.data);
}

return all;
}

/**
* Get array of Label objects from complete list of a Issue's events.
*
Expand Down Expand Up @@ -322,7 +326,6 @@ function params(apiParams, fullRepoName) {
return _.extend({
owner,
repo,
per_page: 100
}, apiParams);
}

Expand Down Expand Up @@ -360,36 +363,31 @@ function parseRepo(repo) {
*/
function getIssueEvents(repo, number) {
debug("Getting events for issue #%s", number);
return rateLimit(github.issues.getEvents)(params({ number }, repo))
.then(paginate);
return github.paginate(githubRest.issues.listEvents, params({ issue_number: number }, repo))
}

function getIssueComments(repo, number) {
debug("Getting comments for issue #%s", number);
return rateLimit(github.issues.getComments)(params({ number }, repo))
.then(paginate);
return github.paginate(githubRest.issues.listComments, params({ issue_number: number }, repo));
}

function getReviews(repo, number) {
debug("Getting reviews for pull #%s", number);
return rateLimit(github.pullRequests.listReviews)(params({ number }, repo))
.then(paginate);
return github.paginate(githubRest.pulls.listReviews, params({ pull_number: number }, repo));
}

function getPullReviewComments(repo, number) {
debug("Getting pull review comments for pull #%s", number);
return rateLimit(github.pullRequests.getComments)(params({ number }, repo))
Comment thread
addison-grant marked this conversation as resolved.
.then(paginate);
return github.paginate(githubRest.pulls.listReviewComments, params({ pull_number: number }, repo))
}

function getCommit(repo, sha) {
return rateLimit(github.repos.getCommit)(params({ sha }, repo))
.then(res => res.data);
return githubRest.repos.getCommit(params({ ref: sha }, repo)).then(res => res.data);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the res.data still needed?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure...I'll look into this.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is still needed because we need the data property off the response. Was there a particular reason you thought we don't need it?

}

function getCommitStatuses(repo, ref) {
debug("Getting commit status for %s", ref);
return rateLimit(github.repos.getCombinedStatusForRef)(params({ ref }, repo))
return githubRest.repos.getCombinedStatusForRef(params({ ref }, repo))
.then(res => res.data.statuses)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the res.data still needed here as well?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, will investigate

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is still needed because we need the particular sub-property off the response. Was there a particular reason you thought we don't need it?

.then(statuses => statuses || [])
}
Expand Down
102 changes: 0 additions & 102 deletions lib/rate-limit.js

This file was deleted.

4 changes: 3 additions & 1 deletion lib/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ var config = require('./config-loader'),

// Set the global Promise object up with the done method so that any
// promise by other libraries will have a .done()
Promise.prototype.done = Bluebird.prototype.done;
Promise.prototype.done = function (callback) {
return Bluebird.cast(this).done(callback);
};
Promise = Bluebird;

module.exports = {
Expand Down
1 change: 1 addition & 0 deletions migrations/0014-pulldasher-reviews-body-nullable.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ALTER TABLE reviews MODIFY body text
Loading