Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Copilot instructions for vscode-java-dependency

## UI and E2E tests

- When asked to add, update, run, or debug UI/E2E coverage, prefer the AutoTest YAML workflow under `test/e2e-plans/`.
- Use the `uitest` skill for UI test work. It should create or update `test/e2e-plans/*.yaml`, validate the plan, build the OSGi bundle and package the extension when needed, run AutoTest, and inspect `test-results/`.
- Do not create legacy VS Code extension tests (`test/maven-suite`, `test/gui`) for UI coverage unless the user explicitly asks for that format.
- Prefer deterministic AutoTest verifiers (`verifyTreeItem`, `verifyFile`, `verifyEditorTab`, `verifyClipboard`) over screenshot-only checks.
49 changes: 49 additions & 0 deletions .github/instructions/uitest-plan.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
applyTo: "test/e2e-plans/**/*.yaml"
description: "Authoring rules for vscode-java-dependency (Project Manager for Java) AutoTest UI/E2E YAML test plans"
---

# AutoTest UI/E2E test plan instructions

Test plans under `test/e2e-plans/` are executable YAML files consumed by `@vscjava/vscode-autotest`. They should describe stable user scenarios for the Java Projects explorer, not raw implementation details.

## Setup rules

- Use `setup.extension: "vscjava.vscode-java-pack"` plus `setup.vscodeVersion: "stable"` for most scenarios. Installing the Extension Pack for Java pulls in every Java extension the Java Projects view relies on, so there is no need to install `redhat.java` separately.
- Install the extension under test from a local VSIX at runtime with `--vsix vscode-java-dependency.vsix` — do not rely on a marketplace copy of `vscjava.vscode-java-dependency`.
- Use existing in-repo fixtures as the workspace: `../maven` (a `maven-archetype-quickstart` project: `my-app` / `com.mycompany.app` / `App.java`) or `../invisible` (an unmanaged-folder project for referenced-library scenarios). Paths are relative to the test plan file. Do not add large binary fixtures.
- Referenced-library / classpath commands (`java.project.addLibraries`, `java.project.removeLibrary`, `java.project.addLibraryFolders`, `java.project.refreshLibraries`) only apply to invisible projects — use `../invisible`, not `../maven`, for those.
- Disable noisy startup surfaces with settings when relevant, for example `workbench.startupEditor: "none"` and `java.configuration.checkProjectSettingsExclusions: false`.

## Action rules

- Prefer stable command IDs via `executeVSCodeCommand` (for example `javaProjectExplorer.focus`, `java.view.package.revealInProjectExplorer`, `workbench.actions.treeView.javaProjectExplorer.collapseAll`) before UI locators. Command IDs are locale-independent.
- Drive the tree with `expandTreeItem <name>` and title-bar buttons with `clickViewTitleAction "Java Projects" "<action>"`. The action resolver only matches the exact `expandTreeItem <name>` form; free-form phrasing silently falls back to the command palette and no-ops.
- Free up sidebar space before asserting tree rows: `executeVSCodeCommand workbench.action.closeAuxiliaryBar`, `collapseSidebarSection OUTLINE`, `collapseSidebarSection TIMELINE`, and `collapseWorkspaceRoot`.
- Use `insertLineInFile` for Java edits that the language server must analyze. Use `typeInEditor` only for text that does not require language-server analysis.
- Use `waitForLanguageServer` before interacting with the tree; prefer verifier polling over long static waits. Short static waits are acceptable only for UI rendering settle time.
- Native file/folder pickers are suppressed in the smoke-test driver; drive VS Code's internal quick-pick with `fillQuickInput` instead of relying on `mockOpenDialog`.
- Quote action arguments that contain spaces:

```yaml
action: 'clickViewTitleAction "Java Projects" "Unlink with Editor"'
```

## Verification rules

- Add deterministic verification to every meaningful step. The natural-language `verify` field is context for humans and failure analysis; it is not pass/fail authority by itself, and it is auto-passed when a plan runs with `--no-llm`.
- Use `verifyTreeItem` (with `name:`, optional `exact: true`, and `visible: false` for absence) as the authoritative check for Java Projects tree state.
- Use `verifyFile` after operations that create, modify, or delete files on disk (new type, export jar, permanent delete). VS Code can open duplicate editor tabs with stale buffers, so prefer file-content checks over editor checks after such operations.
- Use `verifyEditorTab` to assert which file an action opened, and `verifyClipboard` for copy-path commands.
- On state-check steps whose only assertion is a deterministic verifier, omit the `verify:` field to avoid false LLM failures.
- Use screenshots only as diagnostics produced by AutoTest; do not make screenshots the only evidence of pass/fail.

## Local validation commands

```powershell
npx -y @vscjava/vscode-autotest validate test\e2e-plans\<name>.yaml
npm install # first time only; on later iterations run just the commands below
npm run build-server
npx @vscode/vsce package -o vscode-java-dependency.vsix
npx -y @vscjava/vscode-autotest run test\e2e-plans\<name>.yaml --vsix vscode-java-dependency.vsix --no-llm
```
79 changes: 79 additions & 0 deletions .github/skills/uitest/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
name: uitest
description: Write, update, run, or debug vscode-java-dependency (Project Manager for Java) UI/E2E tests using AutoTest YAML plans. Use when the user asks for a UI test, E2E test, VS Code UI validation, Java Projects tree/view test, referenced-library test, or autotest plan.
---

# UI/E2E tests with AutoTest

Use this skill to add or update UI/E2E coverage for `vscode-java-dependency` (Project Manager for Java).

The repository uses `@vscjava/vscode-autotest`: YAML plans in `test/e2e-plans/*.yaml` launch VS Code, install the Extension Pack for Java (`vscjava.vscode-java-pack`) plus a local VSIX of this extension, execute user-facing actions against the Java Projects view, capture screenshots, and write `test-results/<plan>/results.json`.

## Prerequisites (local)

- Node.js >= 18 and JDK 21+ installed and on `PATH` (JDK 21 is required to build the `jdtls.ext` OSGi bundle).
- Close any running VS Code instance before running a plan locally; a running instance can block AutoTest from launching its own VS Code.
- Workspace fixtures are in-repo — no external clones are needed. Plans reference `../maven` (`test/maven`, a `maven-archetype-quickstart` project) or `../invisible` (`test/invisible`, an unmanaged-folder project).

## Workflow

1. Identify the scenario and search `test/e2e-plans/*.yaml` for an existing plan that already covers the area (project explorer, view modes, classpath, export jar, new types, file operations, delete, copy paths, refresh, build lifecycle, autorefresh).
2. Update the existing plan when possible. Create a new `test/e2e-plans/java-dep-<scenario>.yaml` only when no existing plan fits.
3. Use stable AutoTest actions and deterministic verifiers. Do not add raw Playwright tests or screenshot-only checks.
4. Validate the plan:

```powershell
npx -y @vscjava/vscode-autotest validate test\e2e-plans\<name>.yaml
```

5. If validating the current branch, build the OSGi bundle and package the extension:

```powershell
npm install # first time only; on later iterations run just the two commands below
npm run build-server
npx @vscode/vsce package -o vscode-java-dependency.vsix
```

6. Run the plan against the packaged VSIX:

```powershell
npx -y @vscjava/vscode-autotest run test\e2e-plans\<name>.yaml --vsix vscode-java-dependency.vsix --output test-results\<name>
```

Add `--no-llm` to skip natural-language `verify:` fields and rely only on deterministic verifiers for a fast local loop. Run the whole suite with `npm run test-e2e` (`autotest run-all test/e2e-plans --no-llm`).

7. Inspect `test-results/<name>/results.json` and `test-results/<name>/screenshots/`.
8. Iterate based on the failure cause:
- **Incorrect plan**: fix the YAML and rerun step 6. No rebuild is needed.
- **Product code fix**: after editing extension source (`src/**`) or the OSGi bundle (`jdtls.ext/**`), re-run step 5 (rebuild + repackage the VSIX) before rerunning step 6. Never rerun against a stale VSIX.
- **Product bug (report only)**: report the observed behavior and cite the failing step, screenshot, and result reason.

## Authoring rules

- For most plans, use:

```yaml
setup:
extension: "vscjava.vscode-java-pack"
vscodeVersion: "stable"
workspace: "../maven"
settings:
java.configuration.checkProjectSettingsExclusions: false
workbench.startupEditor: "none"
```

- Use `--vsix vscode-java-dependency.vsix` to test current-branch changes; do not rely on a marketplace copy of `vscjava.vscode-java-dependency`.
- Use `../invisible` (not `../maven`) for referenced-library / classpath commands, which only apply to unmanaged-folder projects.
- Prefer `executeVSCodeCommand <commandId>` for command-driven UI (e.g. `javaProjectExplorer.focus`, `java.view.package.revealInProjectExplorer`, `workbench.actions.treeView.javaProjectExplorer.collapseAll`).
- Drive the tree with `expandTreeItem <name>` and title-bar buttons with `clickViewTitleAction "Java Projects" "<action>"`.
- Prefer `verifyTreeItem` for tree state, `verifyFile` for generated/modified/deleted files, `verifyEditorTab` for opened tabs, and `verifyClipboard` for copy-path commands.
- Use `waitForLanguageServer` before tree interactions, and `insertLineInFile` for Java source edits that JDT LS must observe.
- Free sidebar space (`closeAuxiliaryBar`, `collapseSidebarSection OUTLINE`/`TIMELINE`, `collapseWorkspaceRoot`) before asserting tree rows.
- Keep step IDs unique, descriptive, and kebab-case. Omit `verify:` on steps whose only assertion is a deterministic verifier.
- Avoid hard-coded coordinates and brittle DOM structure assumptions.

## CI

The repository workflow `.github/workflows/e2eUI.yml` runs on push and pull requests to `main`. It lints, discovers `test/e2e-plans/*.yaml` into a matrix, builds a branch VSIX per OS, runs every plan on Windows and Linux as independent matrix cells, and uploads `test-results/` artifacts plus an aggregate summary.

Each plan surfaces as its own PR check, so a new `test/e2e-plans/*.yaml` is picked up automatically without editing the workflow.
Loading