add/testing tools orchestration

.claude-plugin/marketplace.json

@@ -102,7 +102,7 @@
     {
       "name": "bitwarden-testing-tools",
       "source": "./plugins/bitwarden-testing-tools",
-      "version": "1.5.0",
+      "version": "1.6.0",
       "description": "Testing tools for Bitwarden — analyzing and improving test quality across its repositories."
     }
   ]

.cspell.json

@@ -177,6 +177,7 @@
     "unreviewed",
     "Unrouted",
     "unsanitized",
+    "uppercased",
     "upserts",
     "urllib",
     "urlparse",

.gitignore

@@ -22,6 +22,7 @@ Thumbs.db
 # Python
 __pycache__/
 *.pyc
+.pytest_cache/
 
 # Node
 node_modules

README.md

@@ -19,7 +19,7 @@ A curated collection of plugins for AI-assisted development at Bitwarden. Enable
 | [bitwarden-product-analyst](plugins/bitwarden-product-analyst/)     | 0.1.7   | Product analyst agent for creating comprehensive Bitwarden requirements documents from multiple sources, and writing user-facing release notes              |
 | [bitwarden-security-engineer](plugins/bitwarden-security-engineer/) | 2.0.0   | Application security engineering: vulnerability triage, threat modeling, and secure code analysis                                                           |
 | [bitwarden-software-engineer](plugins/bitwarden-software-engineer/) | 1.0.0   | Software engineer agent for a Bitwarden product team. Implements stories, tasks, and bugs with code quality, performance, security, and team comms in mind. |
-| [bitwarden-testing-tools](plugins/bitwarden-testing-tools/)         | 1.5.0   | Testing tools for analyzing and improving test quality across Bitwarden's repositories.                                                                     |
+| [bitwarden-testing-tools](plugins/bitwarden-testing-tools/)         | 1.6.0   | Testing tools for analyzing and improving test quality across Bitwarden's repositories.                                                                     |
 | [claude-config-validator](plugins/claude-config-validator/)         | 2.0.2   | Validates Claude Code configuration files for security, structure, and quality                                                                              |
 | [claude-retrospective](plugins/claude-retrospective/)               | 1.1.1   | Analyze Claude Code sessions to identify successful patterns and improvement opportunities                                                                  |
 

package.json

@@ -24,9 +24,10 @@
   },
   "scripts": {
     "prepare": "husky",
-    "lint": "pnpm run lint:prettier && pnpm run lint:spelling",
+    "lint": "pnpm run lint:prettier && pnpm run lint:spelling && pnpm run lint:guardrail",
     "lint:prettier": "prettier --check .",
     "lint:spelling": "cspell lint --no-progress --no-summary --gitignore '**'",
+    "lint:guardrail": "bash plugins/bitwarden-testing-tools/scripts/validate-guardrail.sh",
     "format": "prettier --write ."
   },
   "devEngines": {

plugins/bitwarden-testing-tools/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "bitwarden-testing-tools",
-  "version": "1.5.0",
+  "version": "1.6.0",
   "description": "Testing tools for Bitwarden — analyzing and improving test quality across its repositories.",
   "author": {
     "name": "Bitwarden",

plugins/bitwarden-testing-tools/CHANGELOG.md

@@ -4,6 +4,20 @@ All notable changes to the Bitwarden Testing Tools Plugin will be documented in
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.6.0] - 2026-08-24
+
+### Added
+
+- `start-playwright-test`, the pipeline entry point and the only orchestration skill. It accepts a Jira ticket id, a Jira browse URL, an implementation plan path, or a feature description, optionally followed by extra guidance, plus a `--confirm` flag that pauses for test-plan approval before execution. It runs an eight-task pipeline, dispatching six agents and persisting each response verbatim to `.playwright-testing-artifacts/<slug>/`, then renders an HTML report. Tasks 3 and 4 are dispatched together and run concurrently.
+- Trigger evals for `start-playwright-test`, a 20-query set covering all three input types, the `--confirm` review gate, and near-misses against `assessing-test-coverage` and the separate `qa-testing-notes` skill. Kept as an on-demand diagnostic with no committed baseline; the last observed reading is recorded as dated prose in the eval README.
+- A shared agent non-trigger suite, six queries covering the "do not invoke directly" convention that all six pipeline agents' descriptions carry. Kept as an on-demand diagnostic with no committed baseline; the last observed reading is recorded as dated prose in the eval README.
+- Untrusted-source trust boundary for the web test pipeline: the orchestrator generates a per-run fence token, wraps raw feature source in verified `UNTRUSTED-SOURCE-<nonce>` markers, and prepends a single guardrail to every agent dispatch; each agent carries a one-line backstop, and a `validate-guardrail.sh` check (run in `pnpm lint`) prevents drift.
+
+### Changed
+
+- The plugin README now describes two families of tooling: standalone analysis skills, and the web test pipeline whose components are composed rather than invoked.
+- `scripts/eval_harness.py` counts an `Agent` tool_use as real work alongside `Bash` and `Task`, so the agent non-trigger suite observes a direct agent dispatch.
+
 ## [1.5.0] - 2026-08-24
 
 ### Added