code-complexity

Part 4 Project (SOFTENG 700): Measuring code complexity.

This repository mines Java repositories from GitHub, measures per-method complexity and readability metrics with JavaParser, runs a survey asking programmers which of two code versions is easier to understand, and analyses both in R (and some Python).

Repository layout

code-complexity/
├── tool/                    Maven project: the metric extractor and GitHub harvester
│   ├── config.properties        run mode, search filters, output location
│   ├── src/main, src/test       calculators (CC, cognitive, LOC, nesting, Halstead, ...) and unit tests
│   ├── scripts/                 run_survey.ps1, check_corpus_progress.ps1
│   └── docs/                    metric_definitions.md: how every metric is calculated, and why
├── data/
│   ├── corpus/                  GitHub harvest, October 2026: cc_data.csv, repos.csv, skipped.csv, run_log.txt
│   └── survey/                  Qualtrics definition (.qsf), raw responses, which pair each question showed
├── snippets/                the code shown to survey participants
│   ├── survey_code.java         the 34 snippets, with catalogue and measured metrics
│   ├── images/                  screenshots, side-by-side pair images, the practice question
│   └── variant_selection/       candidate pool before the variants were chosen, ranking spreadsheet
└── analysis/
    ├── report/                  the pipeline that produces every figure, table and number in the report
    │   ├── scripts/                 00_setup.R, 01_load_data, 02_analysis, 03_figures, 04_tables_and_export + run_all.R
    │   └── output/                  generated figures, tables, report_numbers.csv
    ├── survey_stats/            earlier R survey analysis (chi-squared, change classification, ...)
    ├── survey_graphs/           Python survey graphs and isolation analyses
    ├── exploratory/             first corpus analysis (cc_analysis.R, repos_analysis.R) and its output
    └── poster/                  poster script and figures

Detail lives in three READMEs: data/README.md (corpus, survey, snippets), analysis/README.md (all analyses) and tool/docs/metric_definitions.md.

Requirements

• JDK 17+ and Maven (the extractor)
• R 4.x (the report pipeline lists its packages in analysis/README.md; versions used are in analysis/report/output/sessionInfo.txt)
• Python 3 with pandas, numpy, matplotlib, scipy (only for analysis/survey_graphs/)
• A GitHub personal access token in tool/.token (only for the GITHUB_SEARCH and GITHUB_REPO modes; never commit it)

Reproducing the work

1. Harvest the corpus (hours; resumable). Configure tool/config.properties, then from tool/:

mvn compile
mvn exec:java                                          # uses ./config.properties
mvn exec:java -Dexec.args="path/to/other.properties"   # alternate config

Output goes to data/corpus/. Modes: LOCAL_PATH (a file or folder), GITHUB_REPO (one repo), GITHUB_SEARCH (search by filters, clone and analyse in parallel). With output.append=true a restarted harvest skips repos already in repos.csv. Check progress with tool/scripts/check_corpus_progress.ps1.

data/corpus/cc_data.csv is 2 GB and is not in git; it is supplied with the compendium.

2. Measure the survey snippets (from the repo root):

./tool/scripts/run_survey.ps1      # writes snippets/snippet_metrics.csv

3. Run the report analysis (about 3 minutes; empties and regenerates analysis/report/output/):

Rscript analysis/report/scripts/run_all.R

4. Unit tests:

cd tool && mvn test

Survey

The survey asked 75 participants to compare pairs of Java snippets (10 comparisons each, 750 responses in total). Everything about it is in two places: snippets/ (what participants saw) and data/survey/ (what they answered). Detail is in data/README.md.

What	Where
Survey code (the 34 snippets, labelled 01-ORIG, 01-V1, ...)	snippets/survey_code.java
Larger candidate pool from before the final variants were chosen	snippets/variant_selection/survey_code_all_variants_pre_selection.java
Snippet and pair images shown to participants (this is what survey_methods/ became)	snippets/images/screenshots/, snippets/images/pair_images/
Variant rankings spreadsheet (survey_variant_rankings.xlsx)	Removed from the working tree; it is in the git history before the restructure
Measured metrics per snippet	snippets/snippet_metrics.csv
Script that measures the snippets	tool/scripts/run_survey.ps1

Scoring the survey code. tool/scripts/run_survey.ps1 builds the tool and runs SurveyAnalyser on snippets/survey_code.java. It splits the file at each snippet's banner comment (the ID: line), measures every method with the same calculators used for the corpus, prints a table, and writes one row per snippet (its main, first method only) to snippets/snippet_metrics.csv. To score new or edited snippets, keep the banner format, then run:

./tool/scripts/run_survey.ps1

Metrics

The four key metrics the report is built around are cyclomatic complexity (CC), cognitive complexity (CogC), lines of code (LOC) and maximum nesting depth (MaxNest): the cc, cognitive_complexity, loc and max_nesting columns of cc_data.csv. The other measures (Halstead, naming, comments, line length, fan-out and so on) are collected for supporting analysis.

See tool/docs/metric_definitions.md for the definition of each metric and the reasoning behind it. The columns of cc_data.csv are described in data/README.md.

Metric	Source class
Cyclomatic complexity	CyclomaticComplexityVisitor
Cognitive complexity	CognitiveComplexityVisitor
LOC (non-blank, non-comment) and physical	MethodLocCalculator
Nesting depth	MethodNestingCalculator
Identifier / comments	IdentifierMetricsCalculator
Halstead measures	HalsteadCalculator
Maintainability Index	MaintainabilityIndexCalculator
Structure (params, returns, fan-out)	MethodStructureCalculator
Line length	LineLengthCalculator

Notes

• analysis/survey_stats/, analysis/survey_graphs/, analysis/exploratory/ and analysis/poster/ are earlier or supporting analyses. The report numbers come from analysis/report/.
• Some of those scripts still need their input files supplied: survey_graphs/ expects qualtrics_raw.csv and png_metrics_match*.csv beside it, and the poster script reads analysis/survey_stats/700_survey.csv and variant_metrics.csv.
• The live survey loaded its pair images from GitHub main at survey_methods/combinedImages/; those files are now in snippets/images/pair_images/ (earlier draft image sets are in git history).