creating qa skill based on retro feedback

[prograhamming]

🎟️ Tracking

AI-27

📔 Objective

[github-actions]

Plugin Validation Summary — PR #162

Scope: New skill evaluating-qa-readiness added to bitwarden-atlassian-tools.

Files reviewed:

• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/SKILL.md (new)
• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/examples/sample_evaluation.md (new)
• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/references/criteria.md (new)

Overall: ❌ Request changes — one blocking policy violation. The skill itself is structurally sound, secure, and high quality; the PR is missing the mandatory version bump and changelog entry.

---

🔴 Critical (must fix)

1. Missing mandatory version bump + changelog entry for a substantive change
.claude/CLAUDE.md requires that any substantive plugin change — explicitly including adding a new skill — bumps the semantic version across all files and adds a CHANGELOG.md entry, in the same PR. This PR adds a whole new skill but leaves the version untouched:

• plugins/bitwarden-atlassian-tools/.claude-plugin/plugin.json:3 — still "version": "2.2.8"
• .claude-plugin/marketplace.json:51 (the bitwarden-atlassian-tools entry) — still "version": "2.2.8"
• plugins/bitwarden-atlassian-tools/CHANGELOG.md:8 — latest entry is ## [2.2.8] - 2026-07-01 (about pnpm); no entry for this skill

Adding a backward-compatible feature is a MINOR bump: 2.2.8 → 2.3.0.

Remediation:

echo "y" | ./scripts/bump-plugin-version.sh bitwarden-atlassian-tools 2.3.0

(updates plugin.json + marketplace.json; no AGENT.md files exist). Then add to CHANGELOG.md under a new ## [2.3.0] - <date> / ### Added heading, e.g.:

Added evaluating-qa-readiness skill that checks a Jira ticket for the information QA needs before testing (testing instructions, implementation notes, feature-flag state, acceptance criteria, affected clients, linked PR/build) and drafts a developer ask for any gaps.

After bumping, re-run ./scripts/validate-plugin-structure.sh bitwarden-atlassian-tools and ./scripts/validate-marketplace.sh bitwarden-atlassian-tools to confirm name/version consistency.

---

🟠 Major (should fix)

2. Skill description is over-length — SKILL.md:3
The description field is ~817 characters, roughly 60% over the ~500-char guideline. The trigger phrases and proactive-use clause are excellent and should stay; the tail that re-enumerates every criterion (feature flag state, testing instructions, implementation notes, acceptance criteria...) duplicates the SKILL.md body and belongs there, not in the description. A description's job is triggering, not summarizing the workflow.

Remediation: Trim to trigger phrases + one-line purpose + proactive clause (~500 chars); drop the criterion enumeration, which is already in the body.

---

🟡 Minor (optional)

3. Occasional second-person phrasing in Step 2 — SKILL.md:24-28
Phrases like "you gathered" / "you need both" break from the otherwise consistent imperative voice. Acceptable and idiomatic; tighten only if convenient.

4. Criteria summary duplicates the reference — SKILL.md:32-42 vs references/criteria.md
The inline blocking/non-blocking summary intentionally aids the workflow and is short enough to justify keeping. No change needed, but keep the two copies in sync on future edits.

---

✅ Passing checks

Plugin structure (plugin-validator)

• plugin.json valid JSON; name kebab-case, version valid semver, description/author/keywords well-formed; no unknown fields.
• New skill sits correctly under auto-discovered skills/; SKILL.md at skill root; examples/ and references/ present.
• Frontmatter name: evaluating-qa-readiness matches the directory exactly; both required fields present.
• Both referenced files resolve and exist: references/criteria.md (SKILL.md:30), examples/sample_evaluation.md (SKILL.md:93-95).
• .mcp.json untouched and sane — stdio type, secrets injected via ${VAR} env interpolation, ${CLAUDE_PLUGIN_ROOT} used for portability.

Skill quality (skill-reviewer)

• Word counts in ideal range: SKILL.md body ~1,140 words (target 1,000–3,000); criteria.md 762; sample_evaluation.md 519.
• Textbook progressive disclosure: lean core workflow, detailed criterion definitions in references/, complete worked example in examples/, both pointed to from the core.
• Trigger phrases are concrete, quoted, and varied → reliable auto-invocation. Third-person/imperative form is correct.
• Strong "Boundaries and honesty" section (read-only MCP, no fabrication, don't trust ticket status as evidence).

Security (reviewing-claude-config)

• No committed secrets, API keys, tokens, or passwords in any of the three files.
• No settings.local.json, settings files, or permission/auto-approval changes in this PR.
• Only credential-adjacent matches are descriptive product-feature references ("master-password reprompt", "reset-password step") — documentation, not secrets.

---

Bottom line

The new skill is well-designed, secure, and ready on its own merits. Blocking issue: add the 2.3.0 version bump (plugin.json + marketplace.json) and a matching CHANGELOG.md entry in this PR. Recommended: trim the over-length description. Everything else is optional polish.

[SaintPatrck]

Seems like a good start.

[SaintPatrck]

🌱 Write tools are being added in #188 and #189. If leaving comments is ever desired as part of this process, we can add the necessary tool(s) to support that too.

[SaintPatrck]

Oops. Forgot about Claude's review. 🤦

The Critical and Major findings should be addressed before merging.

[github-actions]

Claude Code validation

Result: Issues found

Validated the bitwarden-atlassian-tools plugin and the new evaluating-qa-readiness skill at head c3700dc, against merge base 9afa700. Plugin validation, skill review, and the configuration & security review all ran. The blocking item is an unresolved merge conflict committed into the root README.md by the merge commit on this branch.

Critical

• README.md:13, README.md:17, README.md:22 — Unresolved git merge conflict markers were committed into the plugin catalog table. <<<<<<< HEAD sits at line 13, ======= at line 17, and a prettier-mangled > > > > > > > main at line 22 which has converted the remaining catalog rows (lines 23–32) into a blockquote. The catalog is broken for every plugin, not just this one. Verified directly: no markers exist at the merge base, and git diff 9afa700 HEAD -- README.md shows all three added by this branch's merge commit. Note that prettier has already reformatted the corrupted state, so pnpm run lint will not catch it. Fix: Resolve the conflict by hand — keep exactly one row per plugin, delete the three marker lines, and re-run pnpm run format so the table re-aligns.

Major

• README.md:14 and README.md:18 — bitwarden-atlassian-tools appears twice with conflicting versions: 2.7.1 on the HEAD side and 2.6.0 on the main side. This breaks the four-way version consistency required by .claude/CLAUDE.md (.claude-plugin/marketplace.json = 2.7.1 and plugins/bitwarden-atlassian-tools/.claude-plugin/plugin.json = 2.7.1 both agree; the README catalog does not). This plugin has no agents/ directory, so there is no AGENT.md to bump. Fix: When resolving the conflict, keep the single row at 2.7.1 (the HEAD side, line 14).
• README.md:15-16 — The HEAD side of the conflict carries stale catalog entries for two unrelated plugins, which merging as-is would regress: bitwarden-code-review shows 1.13.1 where marketplace.json says 1.14.0, and bitwarden-delivery-tools shows 3.0.0 with a description missing "Jira ticket filing" where marketplace.json says 3.1.0. Fix: Take the main side (lines 19–20) for these two rows so the catalog matches .claude-plugin/marketplace.json.
• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/SKILL.md:75 — The "not found" state introduced for the Linked PR/build criterion has no corresponding verdict rule and contradicts the rules directly above it. Line 75 says not to "count it toward the verdict as a confirmed gap the way a true Missing would be", but line 70 says "Nearly ready if all blocking criteria pass but one or more non-blocking ones don't" and line 71 says "Ready for QA only if everything passes". A ticket with all blocking criteria present and no discoverable PR satisfies neither branch, so the same ticket can be reported either way. The verdict template at line 55 offers only three strings and requires a gap count (N non-blocking gap(s)) that line 75 forbids producing, and the status legend at line 59 enumerates only ✅ Present / ❌ Missing / ⚠️ Unclear, so the fourth state has no declared symbol. Because the Development panel is invisible to this tooling by design, this is the common case, not an edge case. Flagged independently by both the plugin validator and the skill reviewer. Fix: Add a fourth status to the legend at line 59 (e.g. 🔍 Not verifiable) and a fourth verdict bullet after line 71 stating how it resolves — e.g. "a Not verifiable Linked PR/build does not block Ready for QA; append the caveat to the verdict line instead of counting it as a gap". Mirror the same rule at references/criteria.md:64.

Minor

• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/SKILL.md:23 — The Development-panel limitation is restated in full four times (SKILL.md lines 23, 75, and 83, plus references/criteria.md:62-64) and the feature-flag state-inference rule twice (SKILL.md:39 and references/criteria.md:38), each time with slightly different wording. Four copies of one rule will drift, and they are already not verbatim. Fix: Keep the authoritative statement in references/criteria.md under criterion 6 and reduce SKILL.md to one short mention plus a pointer — the file already does this correctly for the criteria definitions at line 33.
• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/examples/sample_evaluation.md:16 — The single worked example has a PR present via get_issue_remote_links, so it never exercises the PR-not-found path, which is both the subtlest rule in the skill and the behavior this version was specifically correcting per the changelog. Fix: Either change the example so get_issue_remote_links returns nothing (showing the hedged wording in both the table row and the draft comment), or add a second short example covering it.
• plugins/bitwarden-atlassian-tools/CHANGELOG.md:8 — This PR adds a new ### assessing-jira-issue-relevance section to the plugin README (plugins/bitwarden-atlassian-tools/README.md) documenting a skill that already existed at the merge base and was previously undocumented there, but neither the 2.7.1 nor the 2.7.0 entry mentions it — the 2.7.1 README bullet covers only the MCP tool-prefix correction. Fix: Add a bullet under 2.7.1 noting that the previously undocumented assessing-jira-issue-relevance skill is now covered in the plugin README.
• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/SKILL.md:95 — "The Atlassian MCP here is read-only" overstates the case: the bundled server ships create-issue.ts and link-issues.ts, and plugin.json advertises "opt-in Jira write tools". The operative conclusion is still correct (there is no comment-posting tool, and this skill's allowed-tools grant is read-only), but the blanket claim contradicts the plugin's own description. Fix: Narrow the claim to the skill — e.g. "This skill is granted read-only tools, and the server has no comment-posting tool, so you produce the draft for a human to post."
• plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/examples/sample_evaluation.md:37 — The table separator row is wider than every content row, so the example renders as visibly ragged source. Because it sits inside a fenced code block, prettier will not normalize it and pnpm run lint will not catch it. Fix: Re-pad the separator row to match the content column widths.

Notes on scope

Root README.md and .cspell.json were changed by this PR but are not in the supplied Claude-material file list, since neither matches an in-scope path pattern. The three README findings are reported anyway: the plugin catalog table is an explicit version-bump target under .claude/CLAUDE.md, version consistency across those files is owned by plugin validation, and a committed merge conflict is not something a review should stay silent about on a technicality. The .cspell.json change (adding reprompt) is correct and needs no action.

What passed

• Manifest — plugin.json is valid JSON; name is kebab-case and matches the directory; 2.7.1 is valid semver; description, author, and keywords are well-formed.
• Semver appropriateness — 2.6.0 → 2.7.0 for a new skill (MINOR) and 2.7.0 → 2.7.1 for behavioral corrections and a docs fix (PATCH) are both correct. marketplace.json and plugin.json agree at 2.7.1.
• Changelog — Keep a Changelog format, entries present for both 2.7.0 and 2.7.1.
• Skill frontmatter — valid YAML, keys exactly name / description / allowed-tools; name matches the directory; description is 470 characters with three verbatim quoted trigger phrases; correctly omits the deprecated when_to_use.
• Tool grants — all four allowed-tools entries resolve to real registered tools, using the correct plugin-scoped mcp__plugin_bitwarden-atlassian-tools_bitwarden-atlassian__ prefix. The grant is read-only and no wider than the skill needs.
• Progressive disclosure and references — SKILL.md is 1,499 words (target 1,000–3,000); references/criteria.md (1,030) and examples/sample_evaluation.md (517) both exist and every citation resolves.
• Secrets — no hardcoded credentials in any changed file. No settings.local.json in the changeset. The only token-shaped strings are pre-existing your-jira-scoped-token placeholders in the plugin README setup block, untouched here.
• Prompt injection (CWE-1427) — all three new skill files were read as untrusted data. Nothing addresses a reviewer, claims repository-policy authority, or attempts to direct tooling outside the skill's own QA-readiness workflow.
• MCP configuration — .mcp.json is untouched by this changeset.

Checks run

Check	Status
Plugin structure	Skipped — run as a dedicated workflow step before this review; see the job log and check status
Marketplace	Skipped — run as a dedicated workflow step before this review; see the job log and check status
Version bump	Skipped — run as a dedicated workflow step before this review; see the job log and check status
Plugin validation (AI)	Ran — 1 critical, 2 major, 1 minor
Skill review (AI)	Ran — 1 major, 4 minor
Configuration & security	Ran — no findings (covered references/criteria.md and examples/sample_evaluation.md; no CLAUDE.md, .claude/, agent, command, hook, or settings files changed)

[github-actions]

🤖 Bitwarden Claude Code Review

Overall Assessment: REQUEST CHANGES

This PR adds the evaluating-qa-readiness skill to bitwarden-atlassian-tools with a criteria reference and a worked example, bumps the plugin to 2.7.1, and corrects the plugin README's MCP tool prefix to the scoped mcp__plugin_bitwarden-atlassian-tools_bitwarden-atlassian__ form. The skill content itself checks out: frontmatter name matches the directory, both referenced support files exist, allowed-tools names match .mcp.json and the plugin's other skills, only read-only MCP tools are used, and no secrets or settings files are touched. Version bump and changelog entries are present in plugin.json, marketplace.json, and CHANGELOG.md (no agents exist in this plugin). The blocking problem is in the root README.md, which the merge in c3700dc left with unresolved conflict markers.

Not covered: Skill review did not run — this review path cannot launch plugin-dev:skill-reviewer, so plugins/bitwarden-atlassian-tools/skills/evaluating-qa-readiness/SKILL.md was not checked for description quality, length, or progressive disclosure. performing-multi-agent-code-review covers them where plugin-dev is installed.

Code Review Details

• ❌ : Unresolved merge conflict markers committed to the plugin catalog table; 13 of 16 rows no longer render, and the HEAD side would revert bitwarden-code-review to 1.13.1 and bitwarden-delivery-tools to 3.0.0. Lint is green because prettier reflowed the markers into valid markdown, so no CI check catches it.
  • README.md:13-32

[claude]

🎨 SUGGESTED: An empty, clearly-relevant custom field is Missing here but Unclear in the worked example

Details and fix

This line says an empty custom field whose name suggests relevant info is a Missing signal. The worked example hits exactly that case — an empty "Feature Flag" field — and labels it ⚠️ Unclear:

Feature flag → Unclear. A "Feature Flag" field exists but is empty — so it's ambiguous whether the feature is flagged and, if so, what state to test in.

The verdict is unaffected (both statuses fail a blocking criterion), but the two rules assign different labels to identical input, which works against the skill's stated goal that "a developer should be able to look at any 'missing' flag and agree it's genuinely absent."

Suggested fix — have this line defer to the Unclear definition in Step 2:

- If a custom field name suggests it holds relevant info (anything mentioning "flag", "test", "QA", "implementation", "platform") but it's empty, note it by name. A clearly-relevant field left empty is Unclear rather than Missing — the ticket neither answers the question nor disclaims it.

[claude]

❌ CRITICAL: Unresolved merge conflict committed to the root README — 13 of 16 plugin rows no longer render as a table

Details and fix

The merge in c3700dc left conflict markers in the file. pnpm run format then reflowed >>>>>>> main into a markdown blockquote (> > > > > > > main), which pulled the ten rows below it into the same blockquote. That is why Lint is green — prettier sees valid markdown, so no CI check catches this.

Current rendered output:

• <<<<<<< HEAD and ======= appear as literal text on the page.
• Both sides of the conflict render, so bitwarden-atlassian-tools, bitwarden-code-review, and bitwarden-delivery-tools each show twice with different versions.
• The last ten plugins render as a quoted block of raw |-delimited text instead of table rows.

Also note the HEAD side carries stale versions — bitwarden-code-review 1.13.1 and bitwarden-delivery-tools 3.0.0, versus 1.14.0 and 3.1.0 on main. Resolving by keeping HEAD would silently revert both catalog entries, so the resolution needs main's values plus this PR's 2.7.1 for bitwarden-atlassian-tools.

Suggested resolution (run pnpm run format afterward so prettier re-pads the column widths across the whole table):

Suggested change

	<<<<<<< HEAD
	| [bitwarden-atlassian-tools](plugins/bitwarden-atlassian-tools/) | 2.7.1 | Atlassian access via MCP server with deep Jira issue research skill and opt-in Jira write tools |
	| [bitwarden-code-review](plugins/bitwarden-code-review/) | 1.13.1 | Autonomous code review agent following Bitwarden engineering standards with GitHub integration |
	| [bitwarden-delivery-tools](plugins/bitwarden-delivery-tools/) | 3.0.0 | Delivery lifecycle skills: initiative funnel navigation, work transitions, architectural judgment, commits, PRs, preflight, labeling |
	=======
	| [bitwarden-atlassian-tools](plugins/bitwarden-atlassian-tools/) | 2.6.0 | Atlassian access via MCP server with deep Jira issue research skill and opt-in Jira write tools |
	| [bitwarden-code-review](plugins/bitwarden-code-review/) | 1.14.0 | Autonomous code review agent following Bitwarden engineering standards with GitHub integration |
	| [bitwarden-delivery-tools](plugins/bitwarden-delivery-tools/) | 3.1.0 | Delivery lifecycle skills: initiative funnel navigation, work transitions, architectural judgment, commits, PRs, preflight, labeling, Jira ticket filing |
	> > > > > > > main
	> > > > > > > | [bitwarden-designer](plugins/bitwarden-designer/) | 0.1.0 | Product designer persona: Code of Conduct and 30/60/90 critique, critique facilitation; dispatches into bitwarden-design-tools |
	> > > > > > > | [bitwarden-design-tools](plugins/bitwarden-design-tools/) | 0.1.0 | Design toolkit: content style guide, Figma Dev Mode MCP, Bitwarden brand application, handoff prep, Design System governance, Product and Design Jira |
	> > > > > > > | [bitwarden-devops-engineer](plugins/bitwarden-devops-engineer/) | 0.3.0 | DevOps engineering assistant: workflow compliance linting, action security auditing, and org-wide CI/CD remediation |
	> > > > > > > | [bitwarden-init](plugins/bitwarden-init/) | 1.2.2 | Initialize and enhance CLAUDE.md files with Bitwarden's standardized template format |
	> > > > > > > | [bitwarden-product-analyst](plugins/bitwarden-product-analyst/) | 0.1.6 | Product analyst agent for creating comprehensive Bitwarden requirements documents from multiple sources, and writing user-facing release notes |
	> > > > > > > | [bitwarden-security-engineer](plugins/bitwarden-security-engineer/) | 1.3.0 | Application security engineering: vulnerability triage, threat modeling, and secure code analysis |
	> > > > > > > | [bitwarden-software-engineer](plugins/bitwarden-software-engineer/) | 1.0.0 | Software engineer agent for a Bitwarden product team. Implements stories, tasks, and bugs with code quality, performance, security, and team comms in mind. |
	> > > > > > > | [bitwarden-testing-tools](plugins/bitwarden-testing-tools/) | 1.1.0 | Testing tools for analyzing and improving test quality across Bitwarden's repositories. |
	> > > > > > > | [claude-config-validator](plugins/claude-config-validator/) | 2.0.2 | Validates Claude Code configuration files for security, structure, and quality |
	> > > > > > > | [claude-retrospective](plugins/claude-retrospective/) | 1.1.1 | Analyze Claude Code sessions to identify successful patterns and improvement opportunities |
	| [bitwarden-atlassian-tools](plugins/bitwarden-atlassian-tools/) | 2.7.1 | Atlassian access via MCP server with deep Jira issue research skill and opt-in Jira write tools |
	| [bitwarden-code-review](plugins/bitwarden-code-review/) | 1.14.0 | Autonomous code review agent following Bitwarden engineering standards with GitHub integration |
	| [bitwarden-delivery-tools](plugins/bitwarden-delivery-tools/) | 3.1.0 | Delivery lifecycle skills: initiative funnel navigation, work transitions, architectural judgment, commits, PRs, preflight, labeling, Jira ticket filing |
	| [bitwarden-designer](plugins/bitwarden-designer/) | 0.1.0 | Product designer persona: Code of Conduct and 30/60/90 critique, critique facilitation; dispatches into bitwarden-design-tools |
	| [bitwarden-design-tools](plugins/bitwarden-design-tools/) | 0.1.0 | Design toolkit: content style guide, Figma Dev Mode MCP, Bitwarden brand application, handoff prep, Design System governance, Product and Design Jira |
	| [bitwarden-devops-engineer](plugins/bitwarden-devops-engineer/) | 0.3.0 | DevOps engineering assistant: workflow compliance linting, action security auditing, and org-wide CI/CD remediation |
	| [bitwarden-init](plugins/bitwarden-init/) | 1.2.2 | Initialize and enhance CLAUDE.md files with Bitwarden's standardized template format |
	| [bitwarden-product-analyst](plugins/bitwarden-product-analyst/) | 0.1.6 | Product analyst agent for creating comprehensive Bitwarden requirements documents from multiple sources, and writing user-facing release notes |
	| [bitwarden-security-engineer](plugins/bitwarden-security-engineer/) | 1.3.0 | Application security engineering: vulnerability triage, threat modeling, and secure code analysis |
	| [bitwarden-software-engineer](plugins/bitwarden-software-engineer/) | 1.0.0 | Software engineer agent for a Bitwarden product team. Implements stories, tasks, and bugs with code quality, performance, security, and team comms in mind. |
	| [bitwarden-testing-tools](plugins/bitwarden-testing-tools/) | 1.1.0 | Testing tools for analyzing and improving test quality across Bitwarden's repositories. |
	| [claude-config-validator](plugins/claude-config-validator/) | 2.0.2 | Validates Claude Code configuration files for security, structure, and quality |
	| [claude-retrospective](plugins/claude-retrospective/) | 1.1.1 | Analyze Claude Code sessions to identify successful patterns and improvement opportunities |