Skip to content

Update Roster mastery without changing the shared model, and add update_mastery - #97

Open
ChristianMurphy wants to merge 1 commit into
CAHLR:masterfrom
ChristianMurphy:feat/roster-update-mastery-v2
Open

ChristianMurphy wants to merge 1 commit into
CAHLR:masterfrom
ChristianMurphy:feat/roster-update-mastery-v2

Conversation

@ChristianMurphy

@ChristianMurphy ChristianMurphy commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #50. This adds update_mastery, which updates one student's mastery after one answer without refitting or replaying their history, the incremental update #55 and #57 ask for. Roster now uses it, and stops changing the shared model.

pyBKT/fit/update_mastery.py adds update_mastery, a NumPy version update_mastery_array, and transition_mastery for an unobserved attempt. All three read an answer probability of 0, such as a slip of 0 on a wrong answer, as 1. The forward pass in EM_fit.py and the C++ E-step do the same. That keeps the results equal to predict.

Roster had three problems this fixes:

  • It updated a student by swapping the shared model's pi_0 and prior around a predict call, so an update that raised left the model changed.
  • On pure Python, creating a Roster with a multigs model, or adding a student to one, raised a broadcast ValueError. This is Error when using a model trained with multigs=True with the Roster class #50.
  • set_model passed multigs=True where a class name goes, so it failed for multigs models on the C++ build and on pure Python. pyBKT 1.4 made the same mistake when it created a Roster, which is the KeyError: True in How can I save the fitted model? #52.

Roster now steps through the answers with update_mastery, and set_model refreshes each student without passing any class.

Roster.update_state also checks the classes passed to it, on multigs models and on models fit with multilearn. Each of these raises a ValueError naming the skill and the classes it expects:

  • a missing class
  • a class the model wasn't fit with
  • fewer classes than answers

That last case is the question from #78: should update_state reuse one class for a list of answers? This PR says no, which is how guess and slip classes already behave. Reusing it instead would be a small change. Multiprior and multipair models also have extra learn classes, but they still use their Default class for every answer, as on master.

Both new test files compare against predict_onestep. tests/test_roster.py also checks that the model doesn't change, and each new error. On master, 8 of its 15 tests fail on the C++ build and 10 on pure Python.

update_mastery, update_mastery_array and transition_mastery in
pyBKT/fit/update_mastery.py update one student's P(known) after one
answer, or after an unobserved attempt, without refitting or replaying
the history through predict. They apply Bayes' rule on the answer and
then the learn/forget transition, follow the state_predictions
convention, and read an answer probability of exactly 0 as 1, as the
forward pass does.

Roster updated a student by swapping the shared model's pi_0 and prior
around a predict call, with no try/finally, so an update that raised
left the model changed. On the pure-Python build every multigs update
raised a broadcast ValueError. SkillRoster.set_model passed multigs=True
as a class, so it failed for multigs models on both builds (upstream
issue 50). State.predict now steps through the answers with
update_mastery and never writes to the model, and set_model refreshes
each student with no classes.

update_state checks its classes. On a multigs model, and on a model fit
with multilearn, a missing class, a class the model wasn't fit with, or
fewer classes than answers raises ValueError naming the skill and the
expected classes. One class is not reused for a list of answers, the
same rule guess/slip classes follow. Multiprior and multipair models
still use Default for every answer.

tests/test_update_mastery.py and tests/test_roster.py compare the
results with predict_onestep, check that the model doesn't change, and
cover each error.
@sebastiannberg

sebastiannberg commented Oct 23, 2026 •

Copy link
Copy Markdown

thanks for pushing this, api for update mastery looks good. Only thing I would raise is that a -1 in update_mastery(..., -1, ...) is treated as correct but probably should not be?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error when using a model trained with multigs=True with the Roster class

2 participants