Skip to content

Automated Testing: Require changelog entries for all published packages - #83486

Draft
simison wants to merge 1 commit into
trunkfrom
try/require-changelog-every-package
Draft

simison wants to merge 1 commit into
trunkfrom
try/require-changelog-every-package

Conversation

@simison

@simison simison commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Continuation to earlier work to improve changelogs in the repo:

What?

Extends the changelog presence check from a fixed allowlist of packages to every package published to npm. Private packages are skipped.

Consolidates the check into one job in CI, instead separately for each package like today:

CI checks

Why?

Consumers updating packages in their products rely on reading changelogs. Sometimes the apps are running outside WP admin context so "externalization" context is irrelevant to them.

I did this as a proof of concept to see what kind of impact this might have and explore alternatives.

How?

  • Workflow: the check for a changelog entry that cites the PR now covers every package published to npm, not just 16 of them. Private packages are skipped. The check is still optional.
  • Files that don't need an entry:
    • stories and tests
    • Markdown files
    • tsconfig*.json
    • config files at a package's top level (.*rc*, .gitignore, ESLint configs)
  • Docs: packages/README.md and AGENTS.md now describe the new rule. The README still says entries are optional for trivial changes.

Most of these packages don't get changelog entries today, so the check would flag about half of all package PRs, up from 13%.

Replaying the last three months of trunk (1,626 merged PRs that touched packages) through the new filter:

  PRs checked Flagged for a missing entry
Today (16 packages) 527 70 (13%)
New (all published) 1,481 768 (51%)

Packages that already add entries more often include commands (81%), stylelint-config (76%), scripts (66%) and boot (62%).

Packages that would be flagged most often:

Package PRs Had an entry
block-library 445 19%
block-editor 285 42%
editor 216 36%
edit-site 71 36%
eslint-plugin 68 35%
e2e-tests 37 18%

Testing Instructions

CI

Use of AI Tools

yes

Extend the CHANGELOG presence check from a fixed allowlist of packages to every
package published to npm. Private packages are skipped, as are changes that
only touch stories, tests, Markdown files, or development configuration
(tsconfig, top-level tool rc files, .gitignore, and ESLint configs).
@simison
simison requested review from a team and youknowriad September 24, 2026 09:33
@simison simison added [Type] Enhancement A suggestion for improvement. [Status] In Progress Tracking issues with work in progress labels Sep 24, 2026
@simison

simison commented Sep 24, 2026 •

Copy link
Copy Markdown
Member Author

Instead of all packages requiring changelog entries, some learnings from this could make sense to ship:

  • Expand required check to:
  • Ignore more files in the changelog checker (configs, tests) since those are irrelevant to NPM package consumers.
  • Consider if there's real benefit to having multiple CI jobs or if one shared for all packages would be ok; less visual noise in CI checks list.

Alternatively if we would proceed with all packages, we could exclude "internal app" type packages like block-library, block-editor...

@tbradsha tbradsha left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Definitely worth encouraging more changelogs.


{
echo 'packages<<EOF'
echo "$packages"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose a theoretical package called EOF could terminate this early. 😄

# and .gitignore)
# don't need an entry. Private packages aren't published to npm, so
# they're skipped, as are packages removed by the PR.
packages=$(git diff --name-only "$BASE_SHA"...HEAD -- 'packages/' \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might use --no-renames so it doesn't swallow things moving from one package to another.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Status] In Progress Tracking issues with work in progress [Type] Enhancement A suggestion for improvement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants