Repository navigation
Update Roster mastery without changing the shared model, and add update_mastery - #97
Open
ChristianMurphy wants to merge 1 commit into
Open
ChristianMurphy wants to merge 1 commit into
ChristianMurphy wants to merge 1 commit into
Conversation
update_mastery, update_mastery_array and transition_mastery in pyBKT/fit/update_mastery.py update one student's P(known) after one answer, or after an unobserved attempt, without refitting or replaying the history through predict. They apply Bayes' rule on the answer and then the learn/forget transition, follow the state_predictions convention, and read an answer probability of exactly 0 as 1, as the forward pass does. Roster updated a student by swapping the shared model's pi_0 and prior around a predict call, with no try/finally, so an update that raised left the model changed. On the pure-Python build every multigs update raised a broadcast ValueError. SkillRoster.set_model passed multigs=True as a class, so it failed for multigs models on both builds (upstream issue 50). State.predict now steps through the answers with update_mastery and never writes to the model, and set_model refreshes each student with no classes. update_state checks its classes. On a multigs model, and on a model fit with multilearn, a missing class, a class the model wasn't fit with, or fewer classes than answers raises ValueError naming the skill and the expected classes. One class is not reused for a list of answers, the same rule guess/slip classes follow. Multiprior and multipair models still use Default for every answer. tests/test_update_mastery.py and tests/test_roster.py compare the results with predict_onestep, check that the model doesn't change, and cover each error.
This was referenced Oct 22, 2026
|
thanks for pushing this, api for update mastery looks good. Only thing I would raise is that a |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #50. This adds
update_mastery, which updates one student's mastery after one answer without refitting or replaying their history, the incremental update #55 and #57 ask for.Rosternow uses it, and stops changing the shared model.pyBKT/fit/update_mastery.pyaddsupdate_mastery, a NumPy versionupdate_mastery_array, andtransition_masteryfor an unobserved attempt. All three read an answer probability of 0, such as a slip of 0 on a wrong answer, as 1. The forward pass inEM_fit.pyand the C++ E-step do the same. That keeps the results equal topredict.Rosterhad three problems this fixes:pi_0andprioraround a predict call, so an update that raised left the model changed.Rosterwith a multigs model, or adding a student to one, raised a broadcastValueError. This is Error when using a model trained with multigs=True with the Roster class #50.set_modelpassedmultigs=Truewhere a class name goes, so it failed for multigs models on the C++ build and on pure Python. pyBKT 1.4 made the same mistake when it created aRoster, which is theKeyError: Truein How can I save the fitted model? #52.Rosternow steps through the answers withupdate_mastery, andset_modelrefreshes each student without passing any class.Roster.update_statealso checks the classes passed to it, on multigs models and on models fit withmultilearn. Each of these raises aValueErrornaming the skill and the classes it expects:That last case is the question from #78: should
update_statereuse one class for a list of answers? This PR says no, which is how guess and slip classes already behave. Reusing it instead would be a small change. Multiprior and multipair models also have extra learn classes, but they still use theirDefaultclass for every answer, as on master.Both new test files compare against
predict_onestep.tests/test_roster.pyalso checks that the model doesn't change, and each new error. On master, 8 of its 15 tests fail on the C++ build and 10 on pure Python.