Skip to content

#1036: Fix S- and X-learner bootstrap confidence interval reproducibility - #1041

Open
su-jin1425 wants to merge 1 commit into
uber:masterfrom
su-jin1425:fix-issue-1036
Open

#1036: Fix S- and X-learner bootstrap confidence interval reproducibility#1041
su-jin1425 wants to merge 1 commit into
uber:masterfrom
su-jin1425:fix-issue-1036

Conversation

@su-jin1425

Copy link
Copy Markdown
Contributor

Proposed changes

Fixes #1036.

S- and X-learner bootstrap confidence intervals were using the global NumPy random state, making results change when unrelated code consumed the global RNG.

This change:

  • Adds random_state support to the affected S- and X-learner classes.
  • Normalizes the seed with check_random_state.
  • Passes the resulting RNG to the affected bootstrap paths.
  • Ensures random_state is preserved through get_params() and clone().
  • Adds regression tests for reproducible bootstrap confidence intervals.

The existing T-learner seeded bootstrap path and DR-learner behavior are preserved.

Before and After

image

Test

$ErrorActionPreference="Stop"; $tmp="$env:TEMP\causalml-1036"; if(Test-Path $tmp){Remove-Item $tmp -Recurse -Force}; git clone -q "https://github.com/uber/causalml.git" $tmp; Set-Location $tmp; git fetch -q "https://github.com/su-jin1425/causalml.git" "fix-issue-1036"; $env:PYTHONPATH=$tmp; Write-Host "`n========== BEFORE: 9a23278 ==========" -ForegroundColor Yellow; git checkout -q --detach 9a2327811406240925e9b8c0ce16734a04da52d7; @'
import numpy as np
from sklearn.ensemble import RandomForestRegressor
from causalml.inference.meta.slearner import BaseSRegressor
from causalml.inference.meta.xlearner import BaseXRegressor
rng=np.random.RandomState(123); X=rng.normal(size=(80,3)); t=rng.randint(0,2,80); y=2*X[:,0]+t*(1+X[:,1])+rng.normal(size=80); p=np.full(80,t.mean())
def run(M,**kw):
    m=M(learner=RandomForestRegressor(n_estimators=5,random_state=7),**kw)
    return np.asarray(m.estimate_ate(X,y=y,treatment=t,p=p,bootstrap_ci=True,n_bootstraps=30,bootstrap_size=30)[1:])
def check(M,**kw):
    np.random.seed(123); a=run(M,**kw); np.random.rand(10000); b=run(M,**kw); return np.array_equal(a,b)
print("Commit:",__import__("subprocess").check_output(["git","rev-parse","--short","HEAD"],text=True).strip())
print("S reproducible:",check(BaseSRegressor))
print("X reproducible:",check(BaseXRegressor))
'@ | python; Write-Host "`n========== AFTER: 36ce8f2 ==========" -ForegroundColor Green; git checkout -q --detach 36ce8f24672186c4c0ddc23d4ed477d476edcc51; @'
import numpy as np
from sklearn.ensemble import RandomForestRegressor
from causalml.inference.meta.slearner import BaseSRegressor
from causalml.inference.meta.xlearner import BaseXRegressor
rng=np.random.RandomState(123); X=rng.normal(size=(80,3)); t=rng.randint(0,2,80); y=2*X[:,0]+t*(1+X[:,1])+rng.normal(size=80); p=np.full(80,t.mean())
def run(M,**kw):
    m=M(learner=RandomForestRegressor(n_estimators=5,random_state=7),**kw)
    return np.asarray(m.estimate_ate(X,y=y,treatment=t,p=p,bootstrap_ci=True,n_bootstraps=30,bootstrap_size=30)[1:])
def check(M,**kw):
    np.random.seed(123); a=run(M,**kw); np.random.rand(10000); b=run(M,**kw); return np.array_equal(a,b)
print("Commit:",__import__("subprocess").check_output(["git","rev-parse","--short","HEAD"],text=True).strip())
print("S reproducible:",check(BaseSRegressor,random_state=123))
print("X reproducible:",check(BaseXRegressor,random_state=123))
print("Expected: BEFORE False/False | AFTER True/True")
'@ | python; Set-Location $PWD

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

Further comments

The fix uses the existing check_random_state approach established by #1035 and keeps the change limited to bootstrap random-state handling and its regression coverage.

Copilot AI balanced review requested due to automatic review settings August 28, 2026 14:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S-, T- and X-learner bootstrap confidence intervals are not reproducible

2 participants