PR Workflow Evaluation and Interventions [PR]

[jmakowski1123]

Acceptance Criteria

The goal is to arrive at a list of recommendations for where and how the Product WG can intervene to remove bottlenecks in the PR review process and to ensure timely reviews.

Product Brief

https://openedx.atlassian.net/wiki/spaces/OEPM/pages/3476357132/Idea+Board+Project+Briefs#8.-PR-Workflow-Evaluation-and-Interventions

Child Tasks

• Review the steps where product reviews are expected in the OSPR process
• Evaluate the current practice by reviewing the tickets currently needing a product review, and by asking the community for feedback about them
• Trial run
• Improve the product review practice & steps; in particular, figure out a way to ensure contributions don’t end up blocked, along with detection & escalation of any missing product review

[github-actions]

Thanks for your submission, @openedx/open-edx-project-managers will review shortly.

[antoviaque]

@jmakowski1123 @sarina I have started looking into this:

1. Reviewing the OSPR product steps

Review the steps where product reviews are expected in the OSPR process.

As far as I understand, the main page describing the OSPR process and its integration with product review is on the Open edX Developer's Guide - correct @sarina? Are there other places where this is described?

Here are the relevant sections I've spotted in there:

https://edx.readthedocs.io/projects/edx-developer-guide/en/latest/process/community-manager.html

"Help the product team evaluate the idea behind the pull request. Is this something that Open edX wants? If you and the product owner(s) all believe that Open edX does not want this pull request, add a comment to the pull request explaining the reasoning behind that decision. Be polite, and remind them that they are welcome to fork the code and run their own fork on their own servers, without needing permission from edX. Try to suggest ways that they can build something that Open edX does want: for example, perhaps an API that would allow the contributor to build their own component separately. Then close the pull request."

"The community manager should make sure the pull request is ready for Product Review, if that has not yet happened. That means getting enough detail out of the contributor for the product owner to properly do a product review. Once this is done, move the JIRA ticket to the “Product Review” state. If questions arise from product owners during review, work with the contributor to get those questions answered before the next round of review."

_"The product team will meet biweekly to review new proposals and prioritize PRs for team review."

"Once a PR has been prioritized for team review, ask the product owner for an estimate of how many sprints it will take for the pull request to be reviewed: if its more than one, try to push back and advocate for the contributor. However, the estimate is ultimately up to the product owner, and if he/she says it will really be more than one sprint, respect that."

https://edx.readthedocs.io/projects/edx-developer-guide/en/latest/process/product-owner.html

"The product owner has two main responsibilities: approving user-facing features and improvements from a product point of view, and prioritizing pull request reviews.

When a contributor is interested in developing a new feature, or enhancing an existing one, they can engage in a dialogue with the product team about the feature: why it is needed, what does it do, etc. Product owners are expected to fully engage in this process and treat contributors like customers. If the idea is good but the implementation idea is poor, direct them to a better solution. If the feature is not something we can support at this time, provide a detailed explanation of why that is.

Approving work involves more than just giving the idea a go-ahead. There are a number of factors to consider.

• What level of support will edX provide? Unsupported, provisional, or supported?
• Will edx.org use the feature? Does it need configuration support?
• How much documentation is needed for the feature? Will edX write the documentation, or should the contributor provide it?
• Does the work require other review, such as user experience, design, accessibility, internationalization, training, or customer support?

The earlier in the process these other roles are involved, the better the process will work, and the better the final product will be.

A product owner is responsible for prioritizing pull requests from contributors, and keeping them informed when prioritization slips. Pull requests that are ready to be prioritized in the next sprint will have a “Awaiting Prioritization” label on their JIRA review tickets. At every product review meeting (which should happen each sprint), pull requests awaiting prioritization should either be included in the sprint for the appropriate team as a commitment to get the pull request reviewed, or the product owner must inform the author of the pull request that the pull request is still queued and is not being ignored. Contributors should be treated as customers, and if their pull requests are delayed then they should be informed of that, just as a product owner would inform any customer when that customer’s requests are delayed."

2. Review current tickets needing a product review & ask for community feedback

Evaluate the current practice by reviewing the tickets currently needing a product review, and by asking the community for feedback about them.

@sarina Is there a way to list the tickets which are waiting for a product review in Contributions (view) ? It would be useful to be able to keep an eye on that set of tickets.

I know of the following tickets, but there are likely more in the queue:

• Increase grading rounding precision
• Change schedule datetime inputs in studio to user timezone 2

Also, this isn't a PR, but there is the product review of Allow setting due dates for an entire course section which is still pending (there was also PDP-8 for it, but this URL seem to have vanished/moved?). The initial product review request dates back to September 2018. :) Then the PDP was created in 2019 at edX product's request I believe? Not sure if there is a current replacement for this type of product review request.

If we ask for the community's feedback, we might also uncover more cases like this. I could create the forum thread for it.

[sarina]

@antoviaque - let me know if I can answer anything further (we spoke briefly about this at PWG today)

[sarina]

@antoviaque -- Jenna suggests that I spend some time helping you get this ticket to completion. How could I best help out?

[antoviaque]

@sarina @jmakowski1123 Thank you for the help on this! I think one of the important steps currently is to develop a good understanding of how and when the product reviews are conducted currently on OSPRs - that will help to guide figuring out fixes.

@sarina What are in your experience the main blockers, in terms of getting the product reviews assigned, and completed quickly? What are the frequent pain points?

That's actually also related to my next steps/tasks:

• We need to ask for broader community feedback on this, to check whether there is any additional pain points that we haven't identified yet. I have created a feedback thread for this in the official forum, linking back to the current issue for context: https://discuss.openedx.org/t/product-reviews-on-osprs-looking-for-feedback-about-the-process/8152

• During the last product working group meeting, we agreed that it would be useful to get your review on this @natabene - you manage the OSPR board, so you will have unique insights on this, and might already know the answer to points discussed above (like how to display the list of tasks waiting for product review, for example) - would you be able to have a look?

[natabene]

@antoviaque I am happy to help, but I need clarification. Can we chat?

[antoviaque]

@natabene Thank you! And happy to chat yes -- do you want to pick a time in my calendly?

[natabene]

@antoviaque Done