Package information: 
Table of contents:
- Introduction
- Installation
- Input Data
- Using scikit-FIBERS
- Hyperparameters
- Algorithm History
- Citing scikit-FIBERS
- Futher Documentation
- Contact
- Acknowledgements
FIBERS (Feature Inclusion Bin Evolver for Risk Stratification) is an evolutionary machine learning algorithm designed for modeling and/or feature learning in survival data where the 'burden' (i.e. sum) of specific feature values in the dataset may be predictive of a time-to-event outcome (e.g. the burden of specific HLA amino-acid mismatches, between kidney donor and recipient pairs, being predictive of kidney graft failure time). FIBERS can be applied to survival datasets (1) with or without right-censoring, and (2) with or without target covariate features that need to be adjusted for. This implementation is scikit-learn compatible, thus we largely refer to it as scikit-FIBERS.
scikit-FIBERS can be used directly as a modeling strategy, by training a bin population and using the predict() function to apply the discovered bin with the highest fitness as a predictive model of risk group assigment. It can also be used as a feature learning algorithm, by training a bin population and using the transform() function to convert each discovered bin in the population into corresponding dataset features for additional downstream machine learning modeling.
The scikit-FIBERS algorithm seeks to automatically identify and optimize a population of 'candidate bins' that maximize time-to-event differences between high and low risk groups. A 'bin' is a subset of features and an associated 'burden threshold' that together differentiate instances into high vs. low risk instance groups. Instances that have a bin sum (of feature values) greater than the threshold are assigend to the high-risk group, and all others to the low-risk group. The fitness (i.e. quality) of bins in the candidate bin population drives evolutionary algorithm learning.
A schematic detailing how the scikit-FIBERS algorithm works is given below:
scikit-FIBERS currently offers three fitness function options: (1) 'log-rank fitness', for data without covariates, seeks to maximize the separation between high and low-risk survival curves (2) 'residuals fitness', for data with covariates, seeks to maximize the difference between deviance residuals between high and low-risk instance groups, and (3) 'product fitness', for data with covariates, calculates both log-rank and residuals metrics and assignes fitness as the product of both scores.
Further documentation of scikit-FIBERS can be found HERE.
scikit-FIBERS can be installed easily via pip, or by cloning this repository. Either way, make sure that you have also installed all prerequisite packages included in requirements.txt prior to running.
Note: Not yet updated to the most recent release.
pip install scikit-fibers
git clone --single-branch https://github.com/UrbsLab/scikit-FIBERS
cd scikit-FIBERS
pip install -r requirements.txt
scikit-FIBERS takes a pandas dataframe of the target dataset as input (including a header). This dataset can include columns for (1) one or more potentially predictive features (2) a time-to-event outcome, (3) a censoring column (optional), where 0 values indicate that the time-to-event is the censoring time, and 1 values indicate that the time-to-event is the actual observed event time, (4) one or more covariate features (optional) that exclude any perfect correlations among them.
Note that bins sum the feature values of potentially predictive features to determine risk group assignment, so all potentially predictive features should either (1) have the same value range (2) be scaled/normalized ahead of time, and or (3) should have value magnitudes that reflect their weight of importance with respect to other features. All values in the dataset should be numeric.
scikit-FIBERS can also take in a previously trained or manually generated bin population to partially or fully initialize the bin population for training. This bin population is passed as a dataframe of a FIBERS-formatted bin population using the manual_bin_init hyperparameter. This allows users to continue evolving a previously trained bin population for futher iterations, or to use domain/expert knowledge to initialize the evolutionary algorithm with candidate bins.
Two Jupyter Notebooks have been included to demonstrate how scikit-FIBERS (and it's functions) can be applied to data with or without covariates. These demonstrations utilize two survival data simulators that are also included in the scikit-FIBERS repository (i.e. SIM1 and SIM2, for simulating right-censored survival data without or with covariates, respectively).
These notebooks are currently set up to run by downloading this repository and running the respective notebook.
As a simple example of FIBERS training, the following code first loads a hypothetical survival dataset (without covariates) including potentially predictive features, a time-to-event (outcome) column labeled as "Duration", and a censoring indicator column, labeled as "Censoring". Next the FIBERS algorithm is initialized with some basic hyperparameters settings, followed by algorithm training.
train_data = pd.read_csv('my_survival_training_data.csv')
fibers = FIBERS(outcome_label="Duration", iterations=100, pop_size=50, fitness_metric="log_rank", censor_label="Censoring")
fibers = fibers.fit(train_data)
Once trained, FIBERS can be applied to make risk group predictions on a testing dataset that only includes potentially predictive feature columns (i.e. no time-to-event, censoring, or covariate columns). First, the feature columns alone are loaded as a dataframe. Next, FIBERS's predict() function is called with the bin_number parameter set to the bin 'index' in the bin population to be used as a predictive model. Index '0' is the bin with the highest fitness by default. Lastly, assuming we have the true risk groups of each testing instance (saved as a single-column dataframe) we can optionally generate a classification report comparing risk group predictions to true risk group values.
test_data = pd.read_csv('my_survival_testing_data.csv')
predictions = fibers.predict(test_data,bin_number=0)
print(classification_report(predictions, true_risk_group, digits=3))
Lastly, a trained FIBERS bin population can also be converted to learned features in a newly generated dataset using FIBERS transform() function. This feature learning can convert each bin to a new feature by encoding instances as either (1) the sum of bin feature values or (2) the binary risk strata assignment (0=low, 1=high). In the first example below, we transform bins into new features that represent the sum of bin feature values (i.e. full_sums=True) and save this new dataset as a .csv file.
tdf = fibers.transform(train_data,full_sums=True)
tdf.to_csv('my_transformed_dataset_full_sums.csv', index=False)
In this next example, we transform bins into new features that represent the binary risk strata assignment and save this new dataset as a .csv file.
tdf = fibers.transform(train_data,full_sums=False)
tdf.to_csv('my_transformed_dataset_strata.csv', index=False)
While scikit_FIBERS has a number of available hyperparameters only a few are considered to be essential or useful to check or set.
- Essential hyperparameters are given in the first table.
- For survival data without covariate columns, the fitness_metric should be set to 'log_rank' and covariates should be set to None.
- For survival data with covariate colums (that need to be adjusted for), the fitness_metric should be set to either 'residuals' or to 'log_rank_residuals' (otherwise referred to as the 'product' fitness metric), and covariates should be set to a list of all covariate column lables in the dataset.
- For survival data without censoring, censor_label should be set to None, otherwise it should be set the censoring column lable.
- While outcome_type is also an important hyperparameter to check, currently only the 'survival' option (for survial data analysis) has been fully implemented and tested.
| Hyperparameter | Description | Type/Options | Default Value |
|---|---|---|---|
| outcome_label | Data column label for time-to-event (outcome) | str | 'Duration' |
| outcome_type | Defines the type of outcome in the dataset | 'survival','class' | 'survival' |
| censor_label | Data column label for censoring | str/None | 'Censoring', None |
| fitness_metric | Pre-fitness metric driving fitness ranking | 'log_rank','residuals','log_rank_residuals' | 'log_rank' |
| covariates | List of data column labels to be treated as covariates (i.e. not considered for bin inclusion) | list, None | None |
- This second table includes hyperparameters that are not essential but can have a significant impact on algorithm performance.
- In general, setting iterations and pop_size to larger integers is expected to improve training performance, but will require longer run times.
- The group_thresh hyperparameter controls the adaptive burden thresholding in FIBERS, where 'None' activates this mechanism, and an integer value (e.g. 0), will enforce a specific burden threshold for all discovered bins. Related to this, max_thresh controls the maximum burden threshold allowed in bins, assuming group_thresh = None. Also related to adaptive burden thresholding, the thresh_evolve_prob (set between 0.0 and 1.0) can lead to better performance when set closer to 1.0, but requires significantly more runtime.
- The group_strata_min hyperparameter (set as > 0.0 and < 0.5) enforces a minimum instance count balance between risk groups, where 0.5 describes two risk groups with the same instance count.
- The manual_bin_init hyperparameter allows users to load an existing set of candidate bins for FIBERS to start learning from, rather than starting from randomly initialized bins. The value of using this function depends on the quality of the loaded bins, however utilizing expert (i.e. domain) knowledge to design candidate bins before running FIBERS has the potential to dramatically improve or speed up learning in larger or more complex tasks.
- Lastly, pop_clean = 'group_strata' applies a post-hoc cleaning of the bin population, removing any remaining bins that have a risk group instance count ratio below the group_strata_min.
| Hyperparameter | Description | Type/Options | Default Value |
|---|---|---|---|
| iterations | Number of training/optimization cycles | int | 100 |
| pop_size | Maximum bin population size at end of each cycle | int | 50 |
| group_thresh | Optionally specify a group threshold for bins to use | int, None | None |
| max_thresh | Maximum group threshold for adaptive thresholding | int | 5 |
| thresh_evolve_prob | Probability that an optimization cycle will evolve vs. deterministically select a group threshold for new bin evaluation | float | 0.5 |
| group_strata_min | Min. cutoff for group strata sizes below which a pre-fitness penalty is applied to bin | float | 0.2 |
| manual_bin_init | Dataframe of FIBERS-formatted bin population used to initialize the bin population | dataframe, None | None |
| pop_clean | Optional bin population cleanup strategy | 'group_strata', None | None |
- The remaining hyperparameters in the table below can largely be left to their default values by most users.
| Hyperparameter | Description | Type/Options | Default Value |
|---|---|---|---|
| tournament_prop | Population proportion randomly selected for tournament | float | 0.2 |
| crossover_prob | Uniform crossover operator probability | float | 0.5 |
| min_mutation_prob | Minimum mutation operator probability | float | 0.1 |
| max_mutation_prob | Maximum mutation operator probability | float | 0.3 |
| merge_prob | Merge operator probability | float | 0.1 |
| new_gen | Proportion of pop_size. Determines the number of offspring to generate each generation | float | 1.0 |
| elitism | Proportion of pop_size protected from deletion | float | 0.1 |
| diversity_pressure | Number of bin similarity clusters driving bin deletion to encourage bin diversity | int | 3 |
| min_bin_size | Min. number of features to be specified in a bin | int | 1 |
| max_bin_size | Max. number of features to be specified in a bin | int/None | None |
| max_bin_init_size | Max. number of features in initialized bins | int | 10 |
| log_rank_weightning | Optional weighting of log-rank test | 'wilcoxon', 'tarone-ware', 'peto', 'fleming-harrington', None | None |
| penalty | Penalty multiplier applied to pre-fitness when bin’s group strata ratio goes below the minimum | float | 0.5 |
| min_thresh | Minimum group threshold for adaptive thresholding | int | 0 |
| int_thresh | Boolean indicating that adaptive bin thresholds are limited to positive intergers | Boolean | True |
| random_seed | Seed value used to generate random numbers or make random selections | int, None | None |
| report | List of integers, indicating iterations where the population will be printed out for viewing | list, None | None |
| verbose | Boolean flag to run in 'verbose' mode - display run details | Boolean | False |
FIBERS was originally based on the RARE algorithm, an evolutionary algorithm for rare variant binning. (Dasariraju, S. and Urbanowicz, R.J., 2021, July. RARE: evolutionary feature engineering for rare-variant bin discovery. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (pp. 1335-1343).)
The first implementation of FIBERS was developed within it's own GitHub repository, and was applied to an investigation of graft failure in kidney transplantation. (Dasariraju, S., Gragert, L., Wager, G.L., McCullough, K., Brown, N.K., Kamoun, M. and Urbanowicz, R.J., 2023. HLA amino acid Mismatch-Based risk stratification of kidney allograft failure using a novel Machine learning algorithm. Journal of Biomedical Informatics, 142, p.104374.)
The first publication detailing scikit-FIBERS (release 0.9.3) was applied and evaluated on simulated right-censored survival data with amino acid mismatch features. The code for that is available here. (Urbanowicz, R., Bandhey, H., Kamoun, M., Fogarty, N. and Hsieh, Y.A., 2023, July. Scikit-FIBERS: An'OR'-Rule Discovery Evolutionary Algorithm for Risk Stratification in Right-Censored Survival Analyses. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (pp. 1846-1854).) This is the synonmous to FIBERS 1.0 Release.
scikit-FIBERS was extended with a prototype adaptive burden thresholding using 'FIBERS-AT' approach to allow bins to simulaneously identify the best bin threshold to apply. (Bandhey, H., Sadek, S., Kamoun, M. and Urbanowicz, R., 2024, March. Evolutionary Feature-Binning with Adaptive Burden Thresholding for Biomedical Risk Stratification. In International Conference on the Applications of Evolutionary Computation (Part of EvoStar) (pp. 225-239). Cham: Springer Nature Switzerland.)
Most recently scikit-FIBERS 2.1.0 was released, as a completely redesigned, refactored and expanded implementation. Expansions include (1) a merge operator, (2) variable mutation rate, (3) improved adaptive burden thresholding, (4) a bin diversity pressure deletion mechanism, (5) fitness options based on deviance residuals to estimate covariate adjustments throughout algorithm training, (6) a bin population cleanup option, and (7) a number of other helpful functions to report/save the underlying bin population and generate various visualizations. A publication on scikit-FIBERS 2.1.0 is in preparation.
The repository contains high-performance-cluster running scripts to compare previous releases of FIBERS for the performance benchmarking: FIBERS 1.0, FIBERS AT (adaptive thresholding), and FIBERS v2.1.0 (most recent benchmarked release). Code used to apply FIBERS 1.0 and AT are found within https://github.com/UrbsLab/scikit-FIBERS/tree/main/src_archive. Original sources of these earlier releases are as follows:
- FIBERS 1.0: https://github.com/UrbsLab/scikit-FIBERS/tree/gecco_dev / https://github.com/UrbsLab/scikit-FIBERS/releases/tag/v1.0-beta
- FIBERS AT: https://github.com/UrbsLab/scikit-FIBERS/tree/evostar_24
If you use scikit-FIBERS in a scientific publication, please consider citing one of the following papers and indicating the release of scikit-FIBERS utilized (e.g. v2.1.0):
Presently, the manuscript for scikit-FIBERS v2.1.0 is in preparation. In the meantime please consider citing one of the more recent publications and indicate that release scikit-FIBERS v2.1.0 was utilized.
- Evolutionary Feature-Binning with Adaptive Burden Thresholding for Biomedical Risk Stratification. Harsh Bandhey, Sphia Sadek, Malek Kamoun, Ryan J. Urbanowicz (2024).
BibTeX entry:
@inproceedings{bandhey2024evolutionary,
title={Evolutionary Feature-Binning with Adaptive Burden Thresholding for Biomedical Risk Stratification},
author={Bandhey, Harsh and Sadek, Sphia and Kamoun, Malek and Urbanowicz, Ryan},
booktitle={International Conference on the Applications of Evolutionary Computation (Part of EvoStar)},
pages={225--239},
year={2024},
organization={Springer}
}- HLA amino acid Mismatch-Based risk stratification of kidney allograft failure using a novel Machine learning algorithm. Satvik Dasariraju, Loren Gragert, Grace Wager, Nicholas Brown, Malek Kamoun, Ryan J. Urbanowicz (2023). BibTeX entry:
@article{dasariraju2023hla,
title={HLA amino acid Mismatch-Based risk stratification of kidney allograft failure using a novel Machine learning algorithm},
author={Dasariraju, Satvik and Gragert, Loren and Wager, Grace L and McCullough, Keith and Brown, Nicholas K and Kamoun, Malek and Urbanowicz, Ryan J},
journal={Journal of biomedical informatics},
volume={142},
pages={104374},
year={2023},
publisher={Elsevier}
}- Scikit-FIBERS: An 'OR'-Rule Discovery Evolutionary Algorithm for Risk Stratification in Right-Censored Survival Analyses. Harsh Bandhey, Nolan Fogarty, Yi-An Hsieh, Malek Kamoun, Ryan J. Urbanowicz (2023).
BibTeX entry:
@inproceedings{urbanowicz2023scikit,
title={Scikit-FIBERS: An'OR'-Rule Discovery Evolutionary Algorithm for Risk Stratification in Right-Censored Survival Analyses},
author={Urbanowicz, Ryan and Bandhey, Harsh and Kamoun, Malek and Fogarty, Nolan and Hsieh, Yi-An},
booktitle={Proceedings of the Companion Conference on Genetic and Evolutionary Computation},
pages={1846--1854},
year={2023}
}Further code documentation regarding the scikit-FIBERS API can be found here.
Please email Ryan.Urbanowicz@cshs.org or Harsh.Bandhey@cshs.org for any inquiries related to scikit-FIBERS.
The development of FIBERS benefited from feedback across multiple biomedical research collaborators at the University of Pennsylvania, Tulane University, and the Arbor Research Collaborative for Health.
The bulk of the coding for the current version of FIBERS was completed by Ryan Urbanowicz and Harsh Bandhey, with credit to Satvik Dasariraju for his implementation of the original FIBERS 1.0 algorithm. Other algorithm/coding contributions have also made by Nolan Fogarty, Yi-An Hsieh, Sphia Sadek, Brian Ling, Gabe Lipschutz-Villa, and Praneel Vashney.
Funding supporting this work comes from NIH grant: R01 AI173095