Skip to content

Add workshop material lint workflow - #2

Open
cmvcordova wants to merge 2 commits into
mainfrom
add-workshop-lint
Open

cmvcordova wants to merge 2 commits into
mainfrom
add-workshop-lint

Conversation

@cmvcordova

Copy link
Copy Markdown
Member

Moves the lint check trialled in QLS-MiCM/IntroToPython#10 into this repo, so it lives next to the checklist workflow instead of inside one workshop repo. It was prompted by QLS-MiCM/IntroToPython#9, where a missing comma made the notebook invalid JSON and nothing flagged it.

Merging this changes nothing in other repos

  • The distributor workflow is untouched, and its path filter does not match the new template, so nothing is pushed anywhere.
  • The checklist workflow and its caller are untouched.
  • A repo only gets the check when someone adds the caller to it.

What is added

File Purpose
.github/workflows/reusable_lint_workshop.yml Reusable workflow: checks out the PR, takes the linter from this repo, lints the changed files
scripts/lint_workshop.py The linter
.github/workflow-templates/lint_workshop_caller.yml Thin caller to copy into a workshop repo
README.md Documents the above

Unlike the checklist workflow, this one runs on pull_request, with a read-only token and no secrets, which is why it is allowed to check out the PR. The linter always comes from this repo, so a PR cannot edit the checks it is held to.

What it checks

It looks at the files a pull request changes.

Fails the check (the file cannot be opened or run at all):

  • .ipynb is not valid JSON
  • .py, .R or .sh does not parse
  • .Rmd / .qmd has a broken YAML header or a code chunk that is never closed
  • .mlx is not a valid archive

Warns only (annotations and a job summary):

  • a notebook code cell or an Rmd/qmd chunk does not parse
  • a notebook was committed with an error output
  • a notebook uses a relative image path, which does not render in Colab

Cell-level syntax is a warning on purpose: student notebooks contain intentional blanks.

How it was tested

  • Past PRs, locally: the linter was run on the files touched by 29 past pull requests across the organisation (93 files, 16 repos). There were 2 hard failures: IntroToPython#9, and an attendee practice PR in IntroToGitHub (a .R file containing a sentence). None of the 26 merged PRs failed.
  • Each check fires: confirmed on deliberately broken copies of real files.
  • This reusable workflow, on a runner: called from the IntroToPython trial branch, pointing at this branch. It passes on the clean repo in 13 seconds with 9 warnings (run).
  • Failing case, on a runner: tested with the earlier in-repo version of the same script, using the notebook from IntroToPython#9. It fails with line 21: not valid JSON, the notebook cannot be opened (run). The failing case was not re-run through the reusable workflow.

Not covered

  • Nothing is executed, so wrong results in code that runs are not caught.
  • Runners do not ship R. The workflow installs it only when a PR touches .R, .Rmd or .qmd; that step has not run yet, because IntroToPython has no R files.
  • .m files are not checked.

Open questions for rollout

  • Distribution: the distributor handles a single template. Rolling this out to every repo means either generalising it or adding the caller by hand to a few repos first.
  • IntroToGitHub: attendees open practice PRs there, which would fail the check. It probably should not get the caller, or the caller needs a guard.
  • Required or not: as written the check is informative. Making it required is a branch-protection setting per repo.

🤖 Generated with Claude Code

Adds a reusable workflow that lints the files a pull request changes in
a workshop repo, the linter it runs, and a caller template.

Files that cannot be opened or run fail the check: notebooks that are
not valid JSON, Python, R and shell scripts that do not parse, Rmd/qmd
with a broken YAML header or an unclosed chunk. Code cells that do not
parse, committed error outputs and relative image paths are warnings,
since student notebooks contain intentional blanks.

The caller is not distributed: the distributor is unchanged, so merging
this changes nothing in other repos until a repo adds the caller.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Run Rscript with --vanilla so .Rprofile and .Renviron from the repo
  being checked are not loaded. A pull request could ship one that
  quits before parsing, which made broken R files pass.
- Treat an R parser that exits non-zero, or stops before the end, as a
  failed check instead of as clean files.
- Parse notebook cells and Python chunks as plain Python first and only
  strip IPython syntax when that fails. The stripping rewrote text
  inside ordinary strings and comments and produced false warnings,
  for example on print("accuracy = %.2f" % 0.95).

Found by a Codex review of this branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant