Skip to content

Enrollment data handling fixes - #449

Merged
johnbaldwin merged 7 commits into
mainfrom
john/enrollment-data-fixes
Mar 16, 2022
Merged

johnbaldwin merged 7 commits into
mainfrom
john/enrollment-data-fixes

Conversation

@johnbaldwin

@johnbaldwin johnbaldwin commented Mar 11, 2022

Copy link
Copy Markdown
Contributor

This PR has fixes for the new enrollment data processing

Fix backfill_enrollment_data_for_course task.Commit e3faba1

  • Problem was that update_enrollment_data_for_course returns a list of tuples. Each tuple is (object, created), where object is the EnrollmentData record and created tells if the record was created
  • Added logging to the backfill task
  • Updated tests

Fix class for Python27 Ginkgo. Commit bf83408

  • one liner makes test class inherit from object, since we still need to support py27.

Adds enrollments filter method to Figures Course class. Commit bd03842

  • New method figures.course.Course.enrollments_with_student_modules
  • This method finds CourseEnrollment records for a course for enrollments that have StudentModule records in the course. This functionality helps simplify finding out of date EnrollmentData records.

Add test case to tests/test_course.py . Commit: 2566818

  • Was missing the test case when there were no student modules in the course. We want to make sure our "empty set" handling is correct

Add stale course enrollment handling to pipeline. Commit ded1349

  • What's included here is a generator to yield CourseEnrollment records that need to have their EnrollmentData records updated
  • If we install Figures on a site that's already been running or we miss a day or more in processing, then the daily processing will not pick up on all "stale enrollments", enrollments that Figures EnrollmentData record updated.
  • This is because Figures daily metrics jobs only look at "yesterday" in order to work around the limitations of StudentModule.
  • The real logic is in figures.enrollment.is_enrollment_data_out_of_date (Which is still pretty simple)

Fixed figures.tasks.backfill_enrollment_data_for_course. Commit 3511f2b

  • Now it looks for stale course enrollments and for each stale course enrollment found, updates that enrollment's metrics. Prior to this commit, it was just calling the normal daily enrolment data update function.
  • Updated unit tests

Update backfill enrollment management command. Commit cdeb6a2

  • Add ability to load course ids from a file
  • Add test coverage

One last thing to check: The student module GTE check in figures.enrollment.student_modules_for_enrollment_after_date

@johnbaldwin
johnbaldwin force-pushed the john/enrollment-data-fixes branch from e3faba1 to c2796c6 Compare March 11, 2022 19:03
* Problem was that `update_enrollment_data_for_course` returns a list of
tuples. Each tuple is `(object, created)`, where `object` is the
EnrollmentData record and `created` tells if the record was created
* Added logging to the backfill task
* Updated tests
* Added `get_from_enrollment` method to `EnrollmentDataManager`
@johnbaldwin
johnbaldwin force-pushed the john/enrollment-data-fixes branch from c2796c6 to ca245db Compare March 14, 2022 21:04
New method `figures.course.Course.enrollments_with_student_modules`

This method finds `CourseEnrollment` records for a course for
enrollments that have `StudentModule` records in the course. This
functionality helps simplify finding out of date EnrollmentData records.
Was missing the test case when there were no student modules in the
course. We want to make sure our "empty set" handling is correct
What's included here is a generator to yield CourseEnrollment records
that need to have their EnrollmentData records updated

If we install Figures on a site that's already been running or we  miss a day
or more in processing, then the daily processing will not pick up on all
"stale enrollments", enrollments that Figures EnrollmentData record updated.

This is because Figures daily metrics jobs only look at "yesterday" in
order to work around the limitations of StudentModule.

The real logic is in `figures.enrollment.is_enrollment_data_out_of_date`
(Which is still pretty simple)
Now it looks for stale course enrollments and for each stale course
enrollment found, updates that enrollment's metrics. Prior to this
commit, it was just calling the normal daily enrolment data update
function.

* Updated unit tests
@johnbaldwin
johnbaldwin marked this pull request as ready for review March 16, 2022 02:53
* Add ability to load course ids from a file
* Add test coverage
@johnbaldwin
johnbaldwin force-pushed the john/enrollment-data-fixes branch from cdeb6a2 to adb67fb Compare March 16, 2022 02:56
@johnbaldwin

Copy link
Copy Markdown
Contributor Author

Thanks @OmarIthawi !

@johnbaldwin
johnbaldwin merged commit 5261a08 into main Mar 16, 2022
@johnbaldwin
johnbaldwin deleted the john/enrollment-data-fixes branch March 16, 2022 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants