Skip to content

Apply clip_bounds to a user-supplied propensity model - #1043

Open
arpitjain099 wants to merge 1 commit into
uber:masterfrom
arpitjain099:fix/clip-user-propensity
Open

Apply clip_bounds to a user-supplied propensity model#1043
arpitjain099 wants to merge 1 commit into
uber:masterfrom
arpitjain099:fix/clip-user-propensity

Conversation

@arpitjain099

Copy link
Copy Markdown
Contributor

compute_propensity_score documents clip_bounds as keeping the score away from 0 and 1 so later steps do not divide by zero, but the bound only applies when the default model is used. PropensityModel.predict clips internally; a user-supplied classifier goes through predict_proba and its output is returned untouched.

On a separable treatment that saturates:

p_user, _ = compute_propensity_score(X=X, treatment=w, p_model=LogisticRegression(C=1e9, max_iter=1000))
p_def, _ = compute_propensity_score(X=X, treatment=w)
p_user.min(), p_user.max()   # 0.0, 1.0
p_def.min(), p_def.max()     # 0.001, 0.999

Feeding the first one back into the library rejects it, which is what the bound was there to prevent:

BaseDRRegressor(learner=LinearRegression()).fit_predict(X=X, treatment=w, y=y, p=p_user)
AssertionError: The values of p should lie within the (0, 1) interval.

The DR-learner divides by both p_filt and 1 - p_filt, and TMLE by p and 1 - p, so the guard is the only thing standing between a saturated propensity model and an infinity.

The returned score is now clipped whichever model produced it. For the built-in models this is a no-op, since predict has already clipped to the same bounds. BaseRLearner(propensity_learner=...) reaches the unclipped path through self.model_p, so it is covered too.

Two tests: a user-supplied LogisticRegression on separable data has to come back inside the default bounds, which fails on master with assert np.float64(0.0) >= 0.001, and custom bounds have to hold as well.

pytest tests/test_propensity.py tests/test_cate_scoring.py is 39 passing, and black is clean on both files.

Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant