feat(changelog): automate format validation and document process - #363
Conversation
Reformat the existing CHANGELOG.md to strictly follow the Keep a Changelog standard. Add a new section to CONTRIBUTING.md to guide future contributors on the expected update process. Introduce scripts/changelog_validator.py to validate changelog formatting. Fixes OpenAgentHQ#338
There was a problem hiding this comment.
Pull request overview
This PR standardizes the project’s changelog workflow around (an approximation of) the Keep a Changelog structure by updating documentation, adding a changelog validator script, and recording the change in the changelog itself.
Changes:
- Reformats/extends
CHANGELOG.mdunder Unreleased to record the new validation tooling and contributor guidance. - Adds “Changelog Contribution Guidelines” to
CONTRIBUTING.mdso contributors know how to update the changelog. - Introduces
scripts/changelog_validator.pyto validate section headers and basic bullet-list formatting.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| scripts/changelog_validator.py | Adds a validator CLI, but currently has regex/exit-code/message issues that prevent it from reliably validating and may break CI expectations. |
| CONTRIBUTING.md | Documents the changelog process, but needs alignment with what the validator accepts/enforces and consistent command examples. |
| CHANGELOG.md | Adds Unreleased entries describing the new validator and contributor guidance. |
Suppressed comments (4)
scripts/changelog_validator.py:44
validate_entry_format()'ssection_patternuses^but is compiled withoutre.MULTILINE, so it will only ever match at the very start of the file (meaning section entry validation effectively never runs on a normal changelog).
# Pattern: Section Header followed by content that is not indented or bulleted
# We look for a section header, then non-empty lines that do not start with '*' or '-'
section_pattern = re.compile(r"^(#+)\s*(Added|Changed|Fixed|Deprecated|Removed|Breaking)\s*\n(.*?)(?=\n#|\Z)", re.DOTALL)
scripts/changelog_validator.py:56
- The "intro text" allowance logic only checks for a previous line starting with
*, not-. This can produce false positives when a valid-bulleted list is followed by a non-bullet line.
for i, line in enumerate(lines):
if not line.startswith('*') and not line.startswith('-'):
# Allow for potential introductory text, but flag if it's not a list item
if i > 0 and not lines[i-1].startswith('*'):
errors.append(
scripts/changelog_validator.py:75
validate_sections()only fails when no valid section headers exist, butmain()reports "missing required sections" and prints each returned message as if it were a section name. This makes failures and the success message misleading.
missing_sections = validate_sections(content)
if missing_sections:
print("\n[CRITICAL FAILURE] Changelog is missing required sections:", file=sys.stderr)
for section in missing_sections:
print(f" - Missing: {section}", file=sys.stderr)
scripts/changelog_validator.py:88
- The code comment says format inconsistencies are "non-fatal for CI", but
sys.exit(2)will still fail most CI jobs. Either treat warnings as success (exit 0) or update the messaging/CI to explicitly allow exit code 2.
# We treat format warnings as non-fatal for CI, but flag them clearly.
# In a strict gate, this could be sys.exit(1).
sys.exit(2)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Using uv is more conventional for running python scripts. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
🎉 Congratulations @aryansinha1908! Your pull request has been successfully merged into main. 🚀 Thank you for contributing to OpenAgentHQ and helping improve the project. We truly appreciate your contribution and hope to see you back with more amazing PRs! Happy Open Sourcing! ❤️ 🌟 This is your first merged contribution to this repository. |
Reformat the existing CHANGELOG.md to strictly follow the Keep a Changelog standard. Add a new section to CONTRIBUTING.md to guide future contributors on the expected update process. Introduce scripts/changelog_validator.py to validate changelog formatting.
Fixes #338
Description
This PR addresses the manual maintenance overhead and format drift in the changelog by standardizing our process around the Keep a Changelog format.
Changes Made:
h3headers (Added, Changed, Fixed, etc.) and versions use the correcth2format.Type of Change
Related Issues
Closes #338
How Has This Been Tested?
Verified the reformatted
CHANGELOG.mdpasses the newchangelog_validator.pyscript.Tested the validator script against missing sections and malformed bullet points locally to ensure it accurately catches format drift.
Unit tests pass (
uv run pytest)Linter passes (
uv run ruff check .)Type checker passes (
uv run mypy openagent_eval/)Manual testing performed
Checklist
Screenshots (if applicable)
(N/A)
Additional Notes
(N/A)