wip - mooclet remove, native implementation#3223
Draft
danoswaltCL wants to merge 1 commit into
Draft
Conversation
…algorithm, added weight estimation mechanism Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the third-party Mooclet integration and replaces it with a native Thompson Sampling implementation across upgrade_types, backend services/DB, and the Angular dashboard—aiming to preserve the existing UI/UX while switching the underlying adaptive algorithm and reward pipeline to first-party code.
Changes:
- Replaces Mooclet assignment/reward flows with native Thompson Sampling services, entities, repositories, and migrations (including a new rewards summary endpoint).
- Updates frontend models, forms, and experiment details UI to use
thompsonSamplingConfigand show an estimated “weight” via Monte Carlo simulation. - Removes Mooclet toggles, types, env vars, endpoints, and tests/mocks that were specific to the Mooclet service.
Reviewed changes
Copilot reviewed 80 out of 80 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/types/src/Mooclet/MoocletTSConfigurablePolicyParametersDTO.ts | Removed Mooclet TS-configurable DTO types. |
| packages/types/src/Mooclet/MoocletPolicyParametersDTO.ts | Removed Mooclet policy parameter base DTO. |
| packages/types/src/Mooclet/index.ts | Removed Mooclet-related exports/utilities from types. |
| packages/types/src/index.ts | Rewired exports away from Mooclet types to experiment interfaces. |
| packages/types/src/Experiment/interfaces.ts | Added Prior, binary reward types, and reward summary types (incl. estimatedWeight). |
| packages/types/src/Experiment/enums.ts | Renamed adaptive algorithm enum to THOMPSON_SAMPLING. |
| packages/frontend/projects/upgrade/src/environments/environment.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment.staging.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment.qa.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment.prod.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment.local.example.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment.demo.prod.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment.bsnl.ts | Removed moocletToggle. |
| packages/frontend/projects/upgrade/src/environments/environment-types.ts | Updated API endpoint typings; removed Mooclet endpoints/flag. |
| packages/frontend/projects/upgrade/src/assets/i18n/en.json | Updated TS-configurable labels and added “estimated weight” strings. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../ts-configurable-reward-count-table.component.ts | Added tooltip module + estimatedWeight column to displayed columns. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../ts-configurable-reward-count-table.component.scss | Added column sizing for estimated weight column. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../ts-configurable-reward-count-table.component.html | Rendered estimated weight column + tooltip. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../enrollment-condition-expandable-row.component.ts | Switched helper service to Thompson Sampling helper. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../enrollment-condition-expandable-row.component.html | Updated algorithm checks and conditional rendering for TS. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../experiment-details-page-content.component.ts | Simplified “TS experiment” detection to new enum. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../experiment-conditions-table.component.ts | Renamed input flag and switched UI columns for TS. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../experiment-conditions-table.component.html | Updated template checks/labels for TS vs weight-based experiments. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../experiment-conditions-section-card.component.ts | Switched to Thompson Sampling helper and TS config prior path. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/.../experiment-conditions-section-card.component.html | Updated bindings to TS experiment flag + TS priors path. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/modals/upsert-experiment-modal/upsert-experiment-modal.component.ts | Replaced Mooclet child-form state with thompsonSamplingConfig form state. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/modals/upsert-experiment-modal/upsert-experiment-modal.component.html | Rendered TS parameters form based on THOMPSON_SAMPLING selection. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/modals/upsert-experiment-modal/ts-configurable-policy-parameters-form/ts-configurable-policy-parameters-form.component.ts | Refactored form to Thompson Sampling config fields/names. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/modals/upsert-experiment-modal/ts-configurable-policy-parameters-form/ts-configurable-policy-parameters-form.component.html | Renamed formControlName bindings to new TS config names. |
| packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/modals/edit-condition-prior-modal/edit-condition-prior-modal.component.ts | Switched validators provider from Mooclet to TS helper. |
| packages/frontend/projects/upgrade/src/app/core/experiments/thompson-sampling-helper.service.ts | New frontend helper: defaults, validators, formatting, algorithm checks. |
| packages/frontend/projects/upgrade/src/app/core/experiments/store/experiments.selectors.ts | Selector formatting updated to TS config details. |
| packages/frontend/projects/upgrade/src/app/core/experiments/store/experiments.model.ts | Added ThompsonSamplingConfigDTO and renamed labels constants. |
| packages/frontend/projects/upgrade/src/app/core/experiments/store/experiments.effects.ts | Rewards effect now calls the new rewards summary endpoint. |
| packages/frontend/projects/upgrade/src/app/core/experiments/store/experiments.effects.spec.ts | Removed effect tests for rewards fetching (needs replacement). |
| packages/frontend/projects/upgrade/src/app/core/experiments/mooclet-helper.service.ts | Deleted Mooclet helper. |
| packages/frontend/projects/upgrade/src/app/core/experiments/mooclet-helper.service.spec.ts | Deleted Mooclet helper unit tests. |
| packages/frontend/projects/upgrade/src/app/core/experiments/experiments.service.ts | Prior editing now updates thompsonSamplingConfig.priors in frontend state. |
| packages/frontend/projects/upgrade/src/app/core/experiments/experiments.data.service.ts | Added fetchRewardsDataForExperiment(); removed Mooclet rewards method. |
| packages/frontend/projects/upgrade/src/app/core/api-endpoints.constants.ts | Replaced /experiments/mooclet-rewards with /experiments/rewards. |
| packages/backend/test/unit/services/ThompsonSamplingService.test.ts | New unit tests for TS selection + weight estimation. |
| packages/backend/test/unit/services/MoocletRewardsService.test.ts | Removed Mooclet rewards service tests. |
| packages/backend/test/unit/services/MoocletDataService.test.ts | Removed Mooclet data service tests. |
| packages/backend/test/unit/services/ExperimentService.test.ts | Removed Mooclet dependencies from test DI setup. |
| packages/backend/test/unit/services/ExperimentAssignmentService.test.ts | Updated constructor deps; removed Mooclet proxy test. |
| packages/backend/test/unit/controllers/mocks/MoocletRewardsServiceMock.ts | Removed Mooclet rewards mock. |
| packages/backend/test/unit/controllers/mocks/MoocletExperimentServiceMock.ts | Removed Mooclet experiment service mock. |
| packages/backend/test/unit/controllers/ExperimentController.test.ts | Removed Mooclet controller tests/mocks; added TS crud service placeholder. |
| packages/backend/src/types/Mooclet.ts | Removed Mooclet API types. |
| packages/backend/src/env.ts | Removed Mooclet env configuration. |
| packages/backend/src/database/migrations/1781395200000-bootstrapThompsonSamplingConfigs.ts | New bootstrap migration for missing TS configs/posterior rows. |
| packages/backend/src/database/migrations/1781308800000-cleanupMoocletEntities.ts | New migration to drop Mooclet entities + migrate enum values. |
| packages/backend/src/database/migrations/1781222400000-thompsonSamplingEntities.ts | New TS entities/tables + enum updates. |
| packages/backend/src/api/services/ThompsonSamplingService.ts | New core TS sampling + Monte Carlo weight estimation implementation. |
| packages/backend/src/api/services/ThompsonSamplingRewardService.ts | New reward recording + posterior increment service. |
| packages/backend/src/api/services/ThompsonSamplingExperimentCrudService.ts | New CRUD/sync + rewards summary service for TS config/state. |
| packages/backend/src/api/services/MoocletRewardsService.ts | Deleted Mooclet rewards service. |
| packages/backend/src/api/services/MoocletDataService.ts | Deleted Mooclet data service. |
| packages/backend/src/api/services/ImportExportService.ts | Removed Mooclet special-casing for import/export. |
| packages/backend/src/api/services/ExperimentService.ts | Removed Mooclet compatibility validation path. |
| packages/backend/src/api/services/ExperimentAssignmentService.ts | Replaced Mooclet branch with native Thompson Sampling assignment. |
| packages/backend/src/api/repositories/ThompsonSamplingRewardRepository.ts | New repository for raw TS rewards. |
| packages/backend/src/api/repositories/ThompsonSamplingExperimentConfigRepository.ts | New repository for TS experiment config lookups. |
| packages/backend/src/api/repositories/MoocletExperimentRefRepository.ts | Deleted Mooclet experiment ref repository. |
| packages/backend/src/api/repositories/ConditionPosteriorStateRepository.ts | New repository for condition posterior state queries. |
| packages/backend/src/api/models/ThompsonSamplingReward.ts | New entity for TS rewards audit trail. |
| packages/backend/src/api/models/ThompsonSamplingExperimentConfig.ts | New entity for TS config (warmup/minDiff/batchSize). |
| packages/backend/src/api/models/MoocletVersionConditionMap.ts | Deleted Mooclet mapping entity. |
| packages/backend/src/api/models/MoocletExperimentRef.ts | Deleted Mooclet experiment ref entity. |
| packages/backend/src/api/models/ConditionPosteriorState.ts | New entity storing priors + posterior counts per condition. |
| packages/backend/src/api/middlewares/ErrorHandlerMiddleware.ts | Removed Mooclet error type mappings. |
| packages/backend/src/api/errors/MoocletError.ts | Deleted Mooclet error class. |
| packages/backend/src/api/DTO/ExperimentDTO.ts | Replaced Mooclet policy param validation with optional TS config object. |
| packages/backend/src/api/controllers/ExperimentController.ts | Added TS config attach + rewards endpoint; removed Mooclet routing. |
| packages/backend/src/api/controllers/ExperimentClientController.v6.ts | Rewired /v6/reward to record TS rewards. |
| packages/backend/.env.example | Removed Mooclet env var examples. |
| clientlibs/java/src/main/java/org/upgradeplatform/client/QuickTest.java | Updated QuickTest host/context/site/target defaults. |
| CLAUDE.md | Added detailed migration plan/status notes for native Thompson Sampling rollout. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+59
to
+68
| public async updateConfig(experimentId: string, params: ThompsonSamplingConfigParams): Promise<void> { | ||
| await this.configRepository.update( | ||
| { experimentId }, | ||
| { | ||
| warmupThreshold: params.warmupThreshold ?? null, | ||
| minimumDrawDifference: params.minimumDrawDifference ?? null, | ||
| batchSize: params.batchSize ?? null, | ||
| } | ||
| ); | ||
| } |
Comment on lines
+1936
to
+1948
| private async attachThompsonSamplingConfig(experiment: ExperimentDTO): Promise<ExperimentDTO> { | ||
| if (experiment?.assignmentAlgorithm === ASSIGNMENT_ALGORITHM.THOMPSON_SAMPLING) { | ||
| const config = await this.thompsonSamplingCrudService.getConfigForExperiment(experiment.id); | ||
| if (config) { | ||
| experiment.thompsonSamplingConfig = { | ||
| warmupThreshold: config.warmupThreshold, | ||
| minimumDrawDifference: config.minimumDrawDifference, | ||
| batchSize: config.batchSize, | ||
| }; | ||
| } | ||
| } | ||
| return experiment; | ||
| } |
Comment on lines
+82
to
+86
| // Increment counts atomically; successCount only increments on success | ||
| await this.posteriorStateRepository.increment({ id: state.id }, 'totalCount', 1); | ||
| if (success) { | ||
| await this.posteriorStateRepository.increment({ id: state.id }, 'successCount', 1); | ||
| } |
Comment on lines
+505
to
+509
| @IsOptional() | ||
| public thompsonSamplingConfig?: { | ||
| warmupThreshold?: number; | ||
| minimumDrawDifference?: number; | ||
| batchSize?: number; |
| import { MatFormFieldModule } from '@angular/material/form-field'; | ||
| import { MatInputModule } from '@angular/material/input'; | ||
| import { TranslateModule } from '@ngx-translate/core'; | ||
| import { BehaviorSubject, map, Observable, of, startWith, Subject, Subscription } from 'rxjs'; |
Comment on lines
+32
to
+43
| public async down(queryRunner: QueryRunner): Promise<void> { | ||
| // Remove posterior states and configs that were created by this migration. | ||
| // We identify "bootstrapped" rows as those with no warmupThreshold/minimumDrawDifference/batchSize | ||
| // (all nullable, all NULL means they came from this migration with defaults only). | ||
| // This is a best-effort rollback — if configs were subsequently edited, those edits are lost. | ||
| await queryRunner.query(` | ||
| DELETE FROM "thompson_sampling_experiment_config" | ||
| WHERE "warmupThreshold" IS NULL | ||
| AND "minimumDrawDifference" IS NULL | ||
| AND "batchSize" IS NULL | ||
| `); | ||
| } |
Comment on lines
1333
to
1334
| }); | ||
|
|
||
| describe('fetchRewardsDataForExperiment$', () => { | ||
| it('should dispatch actionFetchRewardsDataForExperimentSuccess on successful fetch', fakeAsync(() => { | ||
| const experimentId = 'test-experiment-123'; | ||
| const mockRewardsSummary = [ | ||
| { | ||
| conditionCode: 'Control', | ||
| successes: 10, | ||
| failures: 5, | ||
| total: 15, | ||
| successRate: '66.7%', | ||
| order: 0, | ||
| }, | ||
| { | ||
| conditionCode: 'Treatment', | ||
| successes: 8, | ||
| failures: 7, | ||
| total: 15, | ||
| successRate: '53.3%', | ||
| order: 1, | ||
| }, | ||
| ]; | ||
|
|
||
| experimentDataService.fetchMoocletRewardsDataForExperiment = jest.fn().mockReturnValue(of(mockRewardsSummary)); | ||
|
|
||
| const expectedAction = actionFetchRewardsDataForExperimentSuccess({ | ||
| experimentId, | ||
| rewardsSummary: mockRewardsSummary, | ||
| }); | ||
|
|
||
| service.fetchRewardsDataForExperiment$.subscribe((resultingAction) => { | ||
| expect(resultingAction).toEqual(expectedAction); | ||
| }); | ||
|
|
||
| actions$.next(actionFetchRewardsDataForExperiment({ experimentId })); | ||
|
|
||
| tick(0); | ||
| })); | ||
|
|
||
| it('should dispatch actionFetchRewardsDataForExperimentFailure on fetch error', fakeAsync(() => { | ||
| const experimentId = 'test-experiment-123'; | ||
| const error = new Error('API error'); | ||
|
|
||
| experimentDataService.fetchMoocletRewardsDataForExperiment = jest.fn().mockReturnValue(throwError(error)); | ||
|
|
||
| const expectedAction = actionFetchRewardsDataForExperimentFailure({ error }); | ||
|
|
||
| service.fetchRewardsDataForExperiment$.subscribe((resultingAction) => { | ||
| expect(resultingAction).toEqual(expectedAction); | ||
| }); | ||
|
|
||
| actions$.next(actionFetchRewardsDataForExperiment({ experimentId })); | ||
|
|
||
| tick(0); | ||
| })); | ||
|
|
||
| it('should call experimentDataService with correct experimentId', fakeAsync(() => { | ||
| const experimentId = 'test-experiment-456'; | ||
| const mockRewardsSummary = []; | ||
|
|
||
| experimentDataService.fetchMoocletRewardsDataForExperiment = jest.fn().mockReturnValue(of(mockRewardsSummary)); | ||
|
|
||
| service.fetchRewardsDataForExperiment$.subscribe(() => { | ||
| expect(experimentDataService.fetchMoocletRewardsDataForExperiment).toHaveBeenCalledWith(experimentId); | ||
| }); | ||
|
|
||
| actions$.next(actionFetchRewardsDataForExperiment({ experimentId })); | ||
|
|
||
| tick(0); | ||
| })); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The goal of this change is to remove our dependency on the mooclet API as third-party service, and to instead use a native implementation that does thompson-sampling with similar configurations, with minimal change to UI or topmost backend layers.
The names of certain configurations has changed, but they should do the same thing. Not all configurations were carried over, this assumes binary rewards. This is a re-implementation of the code found here, where the implementation and configurations can be found in the "ts_configurable" policy.
The re-implementation uses no mooclet-based code or external libraries for the critical thompson-sampling algorithm. It uses pure js/typescript implementations for the equivalent of the python packages that provide "beta()" distribution functions for instance. These are javascript implementations that are cited directly from available research.
This also adds a monte-carlo simulation that estimates current weights.|
This is a draft, but is fully functioning and ready to be reviewed and tested.