Apply clip_bounds to a user-supplied propensity model - #1043
Open
arpitjain099 wants to merge 1 commit into
Open
Conversation
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
compute_propensity_scoredocumentsclip_boundsas keeping the score away from 0 and 1 so later steps do not divide by zero, but the bound only applies when the default model is used.PropensityModel.predictclips internally; a user-supplied classifier goes throughpredict_probaand its output is returned untouched.On a separable treatment that saturates:
Feeding the first one back into the library rejects it, which is what the bound was there to prevent:
The DR-learner divides by both
p_filtand1 - p_filt, and TMLE bypand1 - p, so the guard is the only thing standing between a saturated propensity model and an infinity.The returned score is now clipped whichever model produced it. For the built-in models this is a no-op, since
predicthas already clipped to the same bounds.BaseRLearner(propensity_learner=...)reaches the unclipped path throughself.model_p, so it is covered too.Two tests: a user-supplied
LogisticRegressionon separable data has to come back inside the default bounds, which fails on master withassert np.float64(0.0) >= 0.001, and custom bounds have to hold as well.pytest tests/test_propensity.py tests/test_cate_scoring.pyis 39 passing, and black is clean on both files.