This repo contains scripts and utilities for analyzing tandem repeats (TRs).
The latest docker images:
str-analysis python scripts:
weisburd/str-analysis@sha256:07be5f662b9de52499efdf8ccf09fc200b286334d7244b2e5234b664cd6d51f5
scripts + ExpansionHunter:
weisburd/str-analysis-with-expansion-hunter@sha256:9b568cfcad0029abdd77fc37849aaceac014dd4773b50bf6b003547ef44755ac
scripts + TRGT:
weisburd/str-analysis-with-trgt@sha256:5fd565e746b31b605fabce983c33abea12c5ffc5b8560406538031e5b6d4ac42
scripts + ATaRVa:
weisburd/str-analysis-with-atarva@sha256:4deaa82f7808a050bff72ce69af136c269022446523cc22e079948e385bf7039
scripts + inquiSTR:
weisburd/str-analysis-with-inquistr@sha256:27b9cfc86b63a2dcde903377724782b28bfa749412305996405a75ebe5eeb6e5
scripts + HipSTR:
weisburd/str-analysis-with-hipstr@sha256:c9ca655079e465e5c13525a0e397884fbbffd9d13ec394cb392cc488bab62dc3
scripts + GangSTR:
weisburd/str-analysis-with-gangstr@sha256:7a1d54d7ed4a99e8c690a1a17e9a34af0b835c10488f88ceb723dc33824b2406
scripts + EnsembleTR:
weisburd/str-analysis-with-ensembletr@sha256:1287d9c73f094cc36b41b8eedc65618f990e339e40119ee351d06f587cfff0ce
scripts + LongTR:
weisburd/str-analysis-with-longtr@sha256:89f62900811f624be4a7684a5e6a074dced77b0fdd0d1b7dd40793725d3106bc
To install the latest version using pip, run:
python3 -m pip install --upgrade str-analysis@git+https://github.com/broadinstitute/str-analysis
or use the docker image:
docker run -it weisburd/str-analysis:latest
- Variant Filtering & Detection
- Catalog Management & Annotation
- ExpansionHunter Results Processing
- TRGT and LPS Processing
- ExpansionHunterDenovo Post-Processing
- Read Extraction
- Simulation
- gnomAD STR Data Generation
- Catalog Format Conversion
- call_non_ref_motifs - Takes a BAM/CRAM file and, optionally, an ExpansionHunter variant catalog. For each locus, determines which STR motifs are supported by reads overlapping that locus before running ExpansionHunter on the detected motif(s). Useful for detecting non-reference pathogenic motifs (e.g., RFC1). (docs)
- filter_vcf_to_tandem_repeats - Replaces & supersedes filter_vcf_to_STR_variants. This script takes a VCF (from short-read, long-read, or assembly-based callers - it just has to have full REF and ALT sequences) and filters it to insertions and deletions that represent tandem repeat expansions or contractions. It uses both brute-force k-mer search for perfect repeats and Tandem Repeats Finder (TRF) for imperfect repeats, thereby improving detection of VNTRs. It includes 3 subcommands: catalog, merge, genotype (intended to be run in this order). The
catalogsubcommand takes a VCF and a reference FASTA (no TR catalog needed) and detects which insertions and deletions are actually tandem repeat expansions or contractions. It then outputs a BED file of the reference repeat intervals and motifs (a TR catalog) based on what it detected in the VCF.merge(optionally) combines two or more catalogs into a single catalog using some simple heuristics to partially deduplicate redundant entries. Thegenotypesubcommand takes a VCF (not necessarily the same one), a TR catalog BED file and a reference FASTA, and then outputs the diploid TR genotypes at each locus in the catalog derived from the insertion and deletion genotypes in the VCF. - filter_vcf_to_STR_variants - Original version of the above tool. Takes a single-sample VCF file and filters it to the INS/DEL variants that represent tandem repeat expansions or contractions by performing brute-force k-mer search on each variant's inserted or deleted bases. This tool was a core part of Weisburd, B., Tiao, G. & Rehm, H. L. Insights from a genome-wide truth set of tandem repeat variation. (2023)
- parse_motif_composition - Simple algorithm that takes a table of motifs known to occur at a particular VNTR or STR locus with motif variability (such as CACNA1C or RFC1), and also takes a BAM/CRAM data file, and then computes the observed frequency of these motifs in the reads aligned to this locus.
- merge_loci - Takes one or more STR catalogs and combines them into a single catalog while removing duplicates based on overlap and repeat motif.
- annotate_and_filter_str_catalog - Takes an STR catalog and annotates loci based on their overlap with genes and known disease-associated STRs. Allows filtering by motif size, gene region, and various other criteria.
- compute_catalog_stats - Takes an annotated catalog output by annotate_and_filter_str_catalog and computes various summary statistics.
- add_offtarget_regions - Takes an ExpansionHunter variant catalog and adds a list of off-target regions to each locus definition by querying a database of off-target regions precomputed for each TR motif. This database was generated by using wgsim to simulate fully repetitive reads for each motif, and then recording where these reads mapped on hg19 and hg38 after aligning them using bwa.
- add_adjacent_loci_to_expansion_hunter_catalog - Takes an ExpansionHunter variant catalog and a BED file containing all simple repeats in the reference genome. Outputs a new catalog with updated LocusStructures and ReferenceRegions that include any adjacent repeats found near each locus.
- split_adjacent_loci_in_expansion_hunter_catalog - Splits loci with adjacent repeats back into separate loci.
- filter_out_loci_with_Ns_in_flanks - Removes loci from an ExpansionHunter catalog if their flanks contain enough Ns to trigger an ExpansionHunter error.
- maximize_motif_purity - Adjusts motifs to maximize purity for each catalog interval.
- trim_repeat_loci - Trims BED intervals so locus size is a multiple of motif size.
- combine_str_json_to_tsv - Takes a set of ExpansionHunter JSON output files and combines them into a single TSV table.
- combine_json_to_tsv - Takes a set of arbitrary JSON files that share the same schema and combines their top-level fields into a single TSV file.
- postprocess_expansion_hunter_results - Cleans and post-processes ExpansionHunter output (filter hom-ref calls, convert formats).
- filter_to_largest_expansions - Takes a combined table of TR genotypes (from combine_str_json_to_tsv) and filters it to the N most expanded genotypes per locus, based on either the long allele or the short allele (for recessive searches). Optionally filters by quality score, purity, and can discard non-polymorphic loci.
- copy_EH_vcf_fields_to_json - Takes the ExpansionHunter output VCF and JSON file for a given sample and copies fields that are only present in the VCF to the JSON file.
- check_trios_for_mendelian_violations - Takes a table of combined ExpansionHunter calls generated by combine_str_json_to_tsv as well as a FAM or PED file with parent/child relationships, and outputs a table of Mendelian violations in the callset.
- compute_average_genotype_quality_per_sample - Computes average Q score per sample from a genotype table.
- check_combined_results_tsv_for_pathogenic_repeats - Filters combined results for potential pathogenic expansions at known disease loci.
- check_combined_results_tsv_for_genome_wide_repeats - Filters combined results for potential pathogenic expansions genome-wide.
- run_reviewer - Takes ExpansionHunter output files for a single sample and runs REViewer on the subset of loci where the genotypes exceed locus-specific thresholds specified in the variant catalog.
- combine_single_sample_LPS_to_allele_histograms - Takes one or more tables of LPS (longest pure segment) scores and combines them into per-locus allele histograms. Useful for aggregating genotype data across large cohorts.
- convert_multisample_LPS_table_to_allele_frequency_histograms - Takes a wide-format TSV of multisample LPS genotypes (one column per sample, allele sizes as comma-separated pairs) and outputs a per-locus summary table with allele frequency histograms and statistics (min, max, mean, median, mode, stdev, percentiles).
- annotate_EHdn_locus_outliers - Takes an ExpansionHunterDenovo outlier result table (locus outliers or case-control) as well as a BED file containing all simple repeats in the reference genome and, optionally, a gene models GTF file, a variant catalog of known disease-associated loci, and/or other BED files with genomic regions of interest. Outputs a new table where each EHdn outlier is annotated with multiple columns related to the provided reference data.
- convert_annotated_EHdn_locus_outliers_to_expansion_hunter_catalog - Takes the output table from annotate_EHdn_locus_outliers and lets the user apply a range of filters before writing out the passing loci to an ExpansionHunter variant catalog.
- convert_expansion_hunter_denovo_locus_tsv_to_bed - Converts ExpansionHunterDenovo locus TSV results to BED format.
- make_bamlet - Optimized version of ExpansionHunterDenovo's make-bamlet.py. For a given STR region, extracts all relevant reads from a BAM or CRAM file into a much smaller BAMlet which can be used as input to ExpansionHunter instead of the full BAM/CRAM but still yields the same genotype. Reduces I/O operations for better performance.
- make_minicram_for_expansion_hunter - Extracts the minimal CRAM subset needed for ExpansionHunter genotyping.
- print_reads - Extracts reads from CRAM/BAM files overlapping genomic intervals (lightweight alternative to GATK PrintReads).
- simulate_str_expansions - Uses wgsim to generate BAM files with simulated read data containing STR expansions at a given locus, with a specified number of repeats, motif, zygosity, etc.
- generate_gnomad_v2_json - Generates gnomAD-formatted tables and JSON files with readviz metadata. Used to combine the gnomAD STR calls into the files available for download on the gnomAD website. This is the current version of generate_gnomad_json.
- generate_gnomad_json - Original version of generate_gnomad_v2_json (legacy, no longer updated).
- convert_bed_to_expansion_hunter_catalog - Converts BED files to ExpansionHunter variant catalog JSON format.
- convert_expansion_hunter_catalog_to_bed - Converts ExpansionHunter variant catalogs to BED format.
- convert_dat_to_bed - Converts Tandem Repeats Finder (TRF)
.datoutput files to BED format. - convert_expansion_hunter_catalog_to_gangstr_spec - Converts ExpansionHunter catalogs to GangSTR spec format.
- convert_gangstr_spec_to_expansion_hunter_catalog - Converts GangSTR spec to ExpansionHunter variant catalog.
- convert_gangstr_vcf_to_expansion_hunter_json - Converts GangSTR VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_hipstr_format - Converts ExpansionHunter catalogs to HipSTR format.
- convert_hipstr_vcf_to_expansion_hunter_json - Converts HipSTR VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_trgt_catalog - Converts ExpansionHunter catalogs to TRGT catalog format.
- convert_trgt_catalog_to_expansion_hunter_catalog - Converts TRGT repeat catalog BED files to ExpansionHunter variant catalog JSON format.
- convert_trgt_vcf_to_expansion_hunter_json - Converts TRGT VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_longtr_format - Converts ExpansionHunter catalogs to LongTR format.
- convert_expansion_hunter_catalog_to_vamos_catalog - Converts ExpansionHunter catalogs to vamos catalog format.
- convert_straglr_bed_to_expansion_hunter_json - Converts Straglr BED output to ExpansionHunter JSON format.
- convert_strling_calls_to_expansion_hunter_json - Converts STRling calls to ExpansionHunter JSON format.