Skip to content

Create task to process a batch of grade computations. - #14752

Merged
jcdyer merged 1 commit into
masterfrom
neem/compute-task
Mar 28, 2017
Merged

Create task to process a batch of grade computations.#14752
jcdyer merged 1 commit into
masterfrom
neem/compute-task

Conversation

@jcdyer

@jcdyer jcdyer commented Mar 22, 2017

Copy link
Copy Markdown
Contributor

TNL-6690

Create a task to perform computation of grades across a batch of users for a given course.

This leverages CourseGradeFactory.iter(), but adds a "read_only" flag, which we disable for this particular use.

Reviewers

FYI @edx/educator-neem
FYI @jibsheet -- This is the task that will be run by the management command @yro is adding to jenkins.

@jcdyer
jcdyer force-pushed the neem/compute-task branch from 89c65dd to 66178f1 Compare March 27, 2017 20:23

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for @edx/educator-neem devs: It looks like PersistentCourseGrade objects aren't getting created. Is there still a toggle for that which needs to get turned on for the test?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to suggest the write-if-engaged work might be at fault here, but that hasn't merged yet. I'd suggest manually stepping through the code in this test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's working fine now. I think I must have changed something since I updated the test. Not sure what it was.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reintroduced testing the PersistentCourseGrade count.

@jcdyer
jcdyer force-pushed the neem/compute-task branch from 66178f1 to c624396 Compare March 27, 2017 20:43

@efischer19 efischer19 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, assuming the question you asked is resolved.

It seems like this and https://github.com/edx/edx-platform/pull/14771 will generate some merge conflicts depending on which lands first, heads up @nasthagiri

Comment thread lms/djangoapps/grades/tasks.py Outdated
@task(base=_BaseTask)
def compute_grades_for_course(course_key, offset, batch_size): # pylint: disable=unused-argument
"""
TODO: TNL-6690: Fill this task in and remove pylint disables

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pylint disable can be removed from task signature now

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, update this docstring por favor

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done and done.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to suggest the write-if-engaged work might be at fault here, but that hasn't merged yet. I'd suggest manually stepping through the code in this test.

@jcdyer
jcdyer force-pushed the neem/compute-task branch from a91c4ae to a3154b9 Compare March 28, 2017 16:46
@efischer19

Copy link
Copy Markdown
Contributor

🚢 after jenkins is happy

@jcdyer
jcdyer merged commit e9c0017 into master Mar 28, 2017
@jcdyer
jcdyer deleted the neem/compute-task branch March 28, 2017 18:57
@task
def compute_grades_for_course(course_key, offset, batch_size): # pylint: disable=unused-argument
@task(base=_BaseTask)
def compute_grades_for_course(course_key, offset, batch_size):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update this function signature to take kwargs instead? Passing kwargs should now be the best practice for all tasks we create in the future. Otherwise, we always run into backward compatibility issues as @efischer19 and I have encountered. (I was thinking of adding a parameter to this task, related to my PR, but ran into this issue.)
Also, I believe the task isn't retried unless it's encased in a try-except block with a call to self.retry. Please consult @efischer19 on this tomorrow.

@edx-pipeline-bot

Copy link
Copy Markdown
Contributor

EdX Release Notice: This PR has been deployed to the staging environment in preparation for a release to production on Thursday, March 30, 2017.


@ddt.data(*xrange(1, 12, 3))
def test_database_calls(self, batch_size):
per_user_queries = 16 * min(batch_size, 6) # No more than 6 due to offset

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"per_user_queries" is not referenced. I think you meant to reference it on line 413, so I'll make that change in #14731 because I need to up the number of query calls there.

@edx-pipeline-bot

Copy link
Copy Markdown
Contributor

EdX Release Notice: This PR has been deployed to the production environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants