diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..451981cf --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,8 @@ +# Copilot instructions for vscode-java-dependency + +## UI and E2E tests + +- When asked to add, update, run, or debug UI/E2E coverage, prefer the AutoTest YAML workflow under `test/e2e-plans/`. +- Use the `uitest` skill for UI test work. It should create or update `test/e2e-plans/*.yaml`, validate the plan, build the OSGi bundle and package the extension when needed, run AutoTest, and inspect `test-results/`. +- Do not create legacy VS Code extension tests (`test/maven-suite`, `test/gui`) for UI coverage unless the user explicitly asks for that format. +- Prefer deterministic AutoTest verifiers (`verifyTreeItem`, `verifyFile`, `verifyEditorTab`, `verifyClipboard`) over screenshot-only checks. diff --git a/.github/instructions/uitest-plan.instructions.md b/.github/instructions/uitest-plan.instructions.md new file mode 100644 index 00000000..34cc6708 --- /dev/null +++ b/.github/instructions/uitest-plan.instructions.md @@ -0,0 +1,49 @@ +--- +applyTo: "test/e2e-plans/**/*.yaml" +description: "Authoring rules for vscode-java-dependency (Project Manager for Java) AutoTest UI/E2E YAML test plans" +--- + +# AutoTest UI/E2E test plan instructions + +Test plans under `test/e2e-plans/` are executable YAML files consumed by `@vscjava/vscode-autotest`. They should describe stable user scenarios for the Java Projects explorer, not raw implementation details. + +## Setup rules + +- Use `setup.extension: "vscjava.vscode-java-pack"` plus `setup.vscodeVersion: "stable"` for most scenarios. Installing the Extension Pack for Java pulls in every Java extension the Java Projects view relies on, so there is no need to install `redhat.java` separately. +- Install the extension under test from a local VSIX at runtime with `--vsix vscode-java-dependency.vsix` — do not rely on a marketplace copy of `vscjava.vscode-java-dependency`. +- Use existing in-repo fixtures as the workspace: `../maven` (a `maven-archetype-quickstart` project: `my-app` / `com.mycompany.app` / `App.java`) or `../invisible` (an unmanaged-folder project for referenced-library scenarios). Paths are relative to the test plan file. Do not add large binary fixtures. +- Referenced-library / classpath commands (`java.project.addLibraries`, `java.project.removeLibrary`, `java.project.addLibraryFolders`, `java.project.refreshLibraries`) only apply to invisible projects — use `../invisible`, not `../maven`, for those. +- Disable noisy startup surfaces with settings when relevant, for example `workbench.startupEditor: "none"` and `java.configuration.checkProjectSettingsExclusions: false`. + +## Action rules + +- Prefer stable command IDs via `executeVSCodeCommand` (for example `javaProjectExplorer.focus`, `java.view.package.revealInProjectExplorer`, `workbench.actions.treeView.javaProjectExplorer.collapseAll`) before UI locators. Command IDs are locale-independent. +- Drive the tree with `expandTreeItem ` and title-bar buttons with `clickViewTitleAction "Java Projects" ""`. The action resolver only matches the exact `expandTreeItem ` form; free-form phrasing silently falls back to the command palette and no-ops. +- Free up sidebar space before asserting tree rows: `executeVSCodeCommand workbench.action.closeAuxiliaryBar`, `collapseSidebarSection OUTLINE`, `collapseSidebarSection TIMELINE`, and `collapseWorkspaceRoot`. +- Use `insertLineInFile` for Java edits that the language server must analyze. Use `typeInEditor` only for text that does not require language-server analysis. +- Use `waitForLanguageServer` before interacting with the tree; prefer verifier polling over long static waits. Short static waits are acceptable only for UI rendering settle time. +- Native file/folder pickers are suppressed in the smoke-test driver; drive VS Code's internal quick-pick with `fillQuickInput` instead of relying on `mockOpenDialog`. +- Quote action arguments that contain spaces: + +```yaml +action: 'clickViewTitleAction "Java Projects" "Unlink with Editor"' +``` + +## Verification rules + +- Add deterministic verification to every meaningful step. The natural-language `verify` field is context for humans and failure analysis; it is not pass/fail authority by itself, and it is auto-passed when a plan runs with `--no-llm`. +- Use `verifyTreeItem` (with `name:`, optional `exact: true`, and `visible: false` for absence) as the authoritative check for Java Projects tree state. +- Use `verifyFile` after operations that create, modify, or delete files on disk (new type, export jar, permanent delete). VS Code can open duplicate editor tabs with stale buffers, so prefer file-content checks over editor checks after such operations. +- Use `verifyEditorTab` to assert which file an action opened, and `verifyClipboard` for copy-path commands. +- On state-check steps whose only assertion is a deterministic verifier, omit the `verify:` field to avoid false LLM failures. +- Use screenshots only as diagnostics produced by AutoTest; do not make screenshots the only evidence of pass/fail. + +## Local validation commands + +```powershell +npx -y @vscjava/vscode-autotest validate test\e2e-plans\.yaml +npm install # first time only; on later iterations run just the commands below +npm run build-server +npx @vscode/vsce package -o vscode-java-dependency.vsix +npx -y @vscjava/vscode-autotest run test\e2e-plans\.yaml --vsix vscode-java-dependency.vsix --no-llm +``` diff --git a/.github/skills/uitest/SKILL.md b/.github/skills/uitest/SKILL.md new file mode 100644 index 00000000..801bc8ba --- /dev/null +++ b/.github/skills/uitest/SKILL.md @@ -0,0 +1,79 @@ +--- +name: uitest +description: Write, update, run, or debug vscode-java-dependency (Project Manager for Java) UI/E2E tests using AutoTest YAML plans. Use when the user asks for a UI test, E2E test, VS Code UI validation, Java Projects tree/view test, referenced-library test, or autotest plan. +--- + +# UI/E2E tests with AutoTest + +Use this skill to add or update UI/E2E coverage for `vscode-java-dependency` (Project Manager for Java). + +The repository uses `@vscjava/vscode-autotest`: YAML plans in `test/e2e-plans/*.yaml` launch VS Code, install the Extension Pack for Java (`vscjava.vscode-java-pack`) plus a local VSIX of this extension, execute user-facing actions against the Java Projects view, capture screenshots, and write `test-results//results.json`. + +## Prerequisites (local) + +- Node.js >= 18 and JDK 21+ installed and on `PATH` (JDK 21 is required to build the `jdtls.ext` OSGi bundle). +- Close any running VS Code instance before running a plan locally; a running instance can block AutoTest from launching its own VS Code. +- Workspace fixtures are in-repo — no external clones are needed. Plans reference `../maven` (`test/maven`, a `maven-archetype-quickstart` project) or `../invisible` (`test/invisible`, an unmanaged-folder project). + +## Workflow + +1. Identify the scenario and search `test/e2e-plans/*.yaml` for an existing plan that already covers the area (project explorer, view modes, classpath, export jar, new types, file operations, delete, copy paths, refresh, build lifecycle, autorefresh). +2. Update the existing plan when possible. Create a new `test/e2e-plans/java-dep-.yaml` only when no existing plan fits. +3. Use stable AutoTest actions and deterministic verifiers. Do not add raw Playwright tests or screenshot-only checks. +4. Validate the plan: + +```powershell +npx -y @vscjava/vscode-autotest validate test\e2e-plans\.yaml +``` + +5. If validating the current branch, build the OSGi bundle and package the extension: + +```powershell +npm install # first time only; on later iterations run just the two commands below +npm run build-server +npx @vscode/vsce package -o vscode-java-dependency.vsix +``` + +6. Run the plan against the packaged VSIX: + +```powershell +npx -y @vscjava/vscode-autotest run test\e2e-plans\.yaml --vsix vscode-java-dependency.vsix --output test-results\ +``` + + Add `--no-llm` to skip natural-language `verify:` fields and rely only on deterministic verifiers for a fast local loop. Run the whole suite with `npm run test-e2e` (`autotest run-all test/e2e-plans --no-llm`). + +7. Inspect `test-results//results.json` and `test-results//screenshots/`. +8. Iterate based on the failure cause: + - **Incorrect plan**: fix the YAML and rerun step 6. No rebuild is needed. + - **Product code fix**: after editing extension source (`src/**`) or the OSGi bundle (`jdtls.ext/**`), re-run step 5 (rebuild + repackage the VSIX) before rerunning step 6. Never rerun against a stale VSIX. + - **Product bug (report only)**: report the observed behavior and cite the failing step, screenshot, and result reason. + +## Authoring rules + +- For most plans, use: + +```yaml +setup: + extension: "vscjava.vscode-java-pack" + vscodeVersion: "stable" + workspace: "../maven" + settings: + java.configuration.checkProjectSettingsExclusions: false + workbench.startupEditor: "none" +``` + +- Use `--vsix vscode-java-dependency.vsix` to test current-branch changes; do not rely on a marketplace copy of `vscjava.vscode-java-dependency`. +- Use `../invisible` (not `../maven`) for referenced-library / classpath commands, which only apply to unmanaged-folder projects. +- Prefer `executeVSCodeCommand ` for command-driven UI (e.g. `javaProjectExplorer.focus`, `java.view.package.revealInProjectExplorer`, `workbench.actions.treeView.javaProjectExplorer.collapseAll`). +- Drive the tree with `expandTreeItem ` and title-bar buttons with `clickViewTitleAction "Java Projects" ""`. +- Prefer `verifyTreeItem` for tree state, `verifyFile` for generated/modified/deleted files, `verifyEditorTab` for opened tabs, and `verifyClipboard` for copy-path commands. +- Use `waitForLanguageServer` before tree interactions, and `insertLineInFile` for Java source edits that JDT LS must observe. +- Free sidebar space (`closeAuxiliaryBar`, `collapseSidebarSection OUTLINE`/`TIMELINE`, `collapseWorkspaceRoot`) before asserting tree rows. +- Keep step IDs unique, descriptive, and kebab-case. Omit `verify:` on steps whose only assertion is a deterministic verifier. +- Avoid hard-coded coordinates and brittle DOM structure assumptions. + +## CI + +The repository workflow `.github/workflows/e2eUI.yml` runs on push and pull requests to `main`. It lints, discovers `test/e2e-plans/*.yaml` into a matrix, builds a branch VSIX per OS, runs every plan on Windows and Linux as independent matrix cells, and uploads `test-results/` artifacts plus an aggregate summary. + +Each plan surfaces as its own PR check, so a new `test/e2e-plans/*.yaml` is picked up automatically without editing the workflow.