[miniflare] Add local Artifacts binding simulator

.changeset/local-artifacts-binding.md

@@ -0,0 +1,9 @@
+---
+"miniflare": minor
+"wrangler": minor
+"@cloudflare/workers-utils": minor
+---
+
+Add a local Artifacts binding simulator backed by native Git for Worker development
+
+Artifacts bindings now use local, persistent repositories by default in development. Set `remote: true` on the binding to opt into the remote resource. The simulator supports the binding RPC methods and authenticated Git smart HTTP without requiring Cloudflare credentials for local use.

packages/miniflare/scripts/build.mjs

@@ -446,7 +446,13 @@ async function buildPackage() {
 		logLevel: watch ? "info" : "warning",
 		outdir: outPath,
 		outbase: pkgRoot,
-		entryPoints: [indexPath, ...fixtureBuilds],
+		entryPoints: [
+			indexPath,
+			// A separate thread keeps native Git available during synchronous Node
+			// binding-proxy calls, which block Miniflare's main event loop.
+			path.join(pkgRoot, "src/plugins/artifacts/git-sidecar.ts"),
+			...fixtureBuilds,
+		],
 	};
 
 	if (watch) {

packages/miniflare/src/index.ts

@@ -77,6 +77,7 @@ import {
 	WORKER_BINDING_SERVICE_LOOPBACK,
 	WORKFLOWS_PLUGIN_NAME,
 } from "./plugins";
+import { ArtifactsController } from "./plugins/artifacts/controller";
 import { RPC_PROXY_SERVICE_NAME } from "./plugins/assets/constants";
 import { BROWSER_VERSION } from "./plugins/browser-rendering/browser-version";
 import { closeBrowserProcess } from "./plugins/browser-rendering/process";
@@ -888,6 +889,7 @@ export class Miniflare {
 	#maybeInspectorProxyController?: InspectorProxyController;
 	#previousRuntimeInspectorPort?: number;
 
+	#artifactsController = new ArtifactsController();
 	#containerPrivilegesCache = new ContainerPrivilegesCache();
 	#hyperdriveProxyController: HyperdriveProxyController =
 		new HyperdriveProxyController();
@@ -2326,6 +2328,7 @@ export class Miniflare {
 				queueConsumers,
 				containerPrivilegesCache: this.#containerPrivilegesCache,
 				hyperdriveProxyController: this.#hyperdriveProxyController,
+				artifactsController: this.#artifactsController,
 			};
 			for (const [key, plugin] of this.#mergedPluginEntries) {
 				const pluginServicesExtensions = await plugin.getServices({
@@ -2341,6 +2344,11 @@ export class Miniflare {
 					} else {
 						pluginServices = pluginServicesExtensions.services;
 						extensions.push(...pluginServicesExtensions.extensions);
+						for (const socket of pluginServicesExtensions.sockets ?? []) {
+							if (!sockets.some((existing) => existing.name === socket.name)) {
+								sockets.push(socket);
+							}
+						}
 					}
 
 					for (const service of pluginServices) {
@@ -2672,11 +2680,13 @@ export class Miniflare {
 
 	async #assembleAndUpdateConfig(reusePorts = false) {
 		this.#hyperdriveProxyController.beginUpdate();
+		this.#artifactsController.beginUpdate();
 		try {
 			await this.#assembleAndUpdateConfigInternal(reusePorts);
 		} finally {
-			// If assembly or runtime startup failed, keep the old listeners and
-			// close only the new candidates. A successful update already committed.
+			// A failed update retains listeners used by the previous runtime and
+			// closes only candidates started during this attempt.
+			await this.#artifactsController.abortUpdate();
 			this.#hyperdriveProxyController.abortUpdate();
 		}
 	}
@@ -2800,6 +2810,7 @@ export class Miniflare {
 			}
 		}
 		this.#hyperdriveProxyController.commitUpdate(activeExternalAddresses);
+		await this.#artifactsController.commitUpdate();
 		// Note: `updateConfig()` doesn't resolve until ports for all required
 		// sockets have been recorded. At this point, `maybeSocketPorts` contains
 		// all of `requiredSockets` as keys.
@@ -3915,6 +3926,14 @@ export class Miniflare {
 		}
 
 		const runtimeCleanupOutcome = await runtimeDisposeOutcome;
+		try {
+			await this.#artifactsController.dispose();
+		} catch (error) {
+			if (!independentCleanupFailed) {
+				independentCleanupFailed = true;
+				independentCleanupError = error;
+			}
+		}
 		this.#devRegistry.unregisterWorkers();
 		try {
 			await Promise.all(

packages/miniflare/src/plugins/artifacts/LOCAL_DEVELOPMENT.md

@@ -0,0 +1,75 @@
+# Develop with local Artifacts
+
+## Prerequisite: Git
+
+This first version of the local Artifacts emulator requires Git installed globally on your development machine, with `git` available on the PATH of the process starting Wrangler, Vite, or Workers Vitest. Run `git --version` in that environment before starting the dev server or tests. If Git is missing or cannot run, local Artifacts fails at startup with installation instructions instead of waiting for a repository operation. Restart your dev tool after installing Git so it picks up the updated PATH.
+
+A binding with `"remote": true` does not start the local emulator and does not need local Git. Production Artifacts also does not require Git on your machine. We plan to open-source a JavaScript Git engine in the future; it is not part of this initial local release, which uses native Git.
+
+## Configure a binding
+
+For a Wrangler project, add an Artifacts binding to `wrangler.jsonc`:
+
+```jsonc
+{
+	"name": "example-worker",
+	"main": "src/index.ts",
+	"compatibility_date": "2026-09-03",
+	"artifacts": [{ "binding": "REPOS", "namespace": "examples" }],
+}
+```
+
+To try this unreleased feature from the Workers SDK checkout, install its dependencies and build Miniflare and Wrangler, then start the checkout's CLI from the app directory:
+
+```sh
+# From the workers-sdk root:
+pnpm install --frozen-lockfile
+pnpm --filter miniflare build
+pnpm --filter wrangler build
+
+# From your app directory, using the absolute path to this checkout:
+node /path/to/workers-sdk/packages/wrangler/bin/wrangler.js dev --port=0 --inspector-port=0
+```
+
+Open the URL on Wrangler's `Ready on` line. With `@cloudflare/vite-plugin`, use the same Wrangler config with `cloudflare()` in your Vite config and start the Vite dev server. In Workers Vitest, point `cloudflareTest({ wrangler: { configPath: "./wrangler.jsonc" } })` at this config. These paths share the Miniflare implementation. To force local behavior explicitly, add `"remote": false` to the binding. The absence of `remote` is also local; this differs from older development behavior that implicitly used a remote Artifacts resource. Do not rely on production repositories being available locally.
+
+A Worker can create and inspect a local repository:
+
+```ts
+const created = await env.REPOS.create("demo");
+const repo = await env.REPOS.get("demo");
+const listing = await env.REPOS.list();
+const info = await repo.info();
+// created.remote is a loopback HTTP Git URL; created.token is a secret.
+```
+
+To use Git smart HTTP, use the returned `remote` URL and a repository token. If `git clone` prompts for a username and password, enter any username (for example, `git`) and use `created.token` as the password. This is an Artifacts repository token, **not** a GitHub or Cloudflare account password. A configured Git credential helper may remember a password entered at the prompt. To avoid that, supply a bearer token for one Git command without storing it in source or in a Git remote URL:
+
+```sh
+read -s REPO_TOKEN # Paste the token returned by create(); input is hidden.
+printf '\n'
+# Replace the URL with the `remote` value returned by create() or info().
+GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.extraHeader \
+  GIT_CONFIG_VALUE_0="Authorization: Bearer $REPO_TOKEN" \
+  git clone 'http://127.0.0.1:<port>/git/examples/demo.git'
+unset REPO_TOKEN
+```
+
+After pushing `hello.txt` to `main`, read it in the Worker with `await repo.readFile({ ref: "main", path: "hello.txt" })` (a `Blob`, or `null` when absent). The initial create token permits writes. Its plaintext is returned only once; if you lose it, use `repo.createToken("read")` for cloning or `repo.createToken("write")` for pushing and save the returned `plaintext` securely. Revoke tokens that are no longer needed. Avoid printing tokens or committing them to config. A plain request without a token cannot clone a private local repository. Git's URL uses an ephemeral loopback port; obtain the current URL after restarting the dev server instead of hardcoding it.
+
+## Storage and network boundaries
+
+Local repositories and metadata persist under the dev tool's Miniflare resource persistence path (for example, Wrangler's `.wrangler/state/` when persistence is enabled). Use the same project root and persistence settings to keep local data across reloads and restarts. Disabling persistence (for example, Vite's `persistState: false`) makes the data temporary. Local and deployed Artifacts are separate: local create, push, and delete operations do not update remote resources. To opt in to a remote binding, set `"remote": true` on that Artifacts binding and configure the credentials required by your development tool. Remote mode uses a Cloudflare proxy, not the local Git store; do not assume local data migrates to it.
+
+To reset local state, stop the dev server first, back up any repositories you need, then delete the `artifacts/` directory inside that dev tool's resource persistence path. Older draft persisted Artifacts layouts are not automatically migrated and may require this backup-and-reset procedure. Do not delete unrelated binding data in the same persistence directory.
+
+A local binding does not contact Cloudflare for repository operations. **Explicit imports from HTTPS Git sources are different:** they make outbound network requests from your development machine to the supplied host, including private-network hosts. Import does not follow HTTP redirects; supply the final HTTPS Git URL so credentials cannot be redirected to another destination. Imports can require authentication and may fail due to TLS certificates, connectivity, or upstream Git permissions. For a private Git CA, set `GIT_SSL_CAINFO` to the trusted CA file in the dev tool's host environment before starting the dev server. Do not disable certificate verification. After a successful import, URL credentials are omitted from the local repository metadata and Git config. In-flight process-crash recovery is not yet validated; treat partial local storage after an interrupted import as sensitive if the URL contained credentials. Treat the import URL as a secret while calling `import()` and avoid logging it. Avoid importing untrusted or private URLs in examples or tests. No remote resources are contacted by the local-only examples above.
+
+## Troubleshooting and limits
+
+- If startup says local Artifacts requires Git, install Git globally, check `git --version` in the dev tool's environment, and restart the tool with `git` on PATH.
+- If startup rejects an older storage layout, back it up and reset only the Artifacts persistence directory; a restart alone will not convert it.
+- If Git asks for a username and password, use any username and the repository token as the password. If Git reports unauthorized, check the repository, scope, expiry, and revocation; do not use a Cloudflare or GitHub account password.
+- If startup reports `Address already in use`, the dev or inspector port may be occupied. Pass `--port=0 --inspector-port=0` and use the printed `Ready on` URL.
+- If a Git URL stops responding after restart, get the new `remote` from the binding; the loopback listener is tied to the dev process.
+- Local development is not a replica of production placement or infrastructure. The local Git endpoint is HTTP on loopback only, and local state and tokens do not transfer to a remote namespace. Miniflare Artifacts, Wrangler dev, and Vite local fixtures have passed Windows CI; Workers Vitest's Artifacts fixture has not been verified on Windows.

packages/miniflare/src/plugins/artifacts/README.md

@@ -0,0 +1,148 @@
+# Artifacts in Miniflare
+
+For setup, persistence, Git authentication, and local/remote behavior, see
+[Local development](./LOCAL_DEVELOPMENT.md).
+
+The Artifacts plugin implements the local Artifacts binding in workerd. This
+first version requires Git installed globally on the host and available on
+PATH; Miniflare checks at local startup and gives installation instructions if
+Git is missing. A Node worker thread runs the native Git backend. Unless
+`dev.remote` is explicitly `true`, an Artifacts binding uses local storage
+rather than contacting Cloudflare. We plan to open-source a JavaScript Git
+engine in the future; it is not included in this version.
+
+## Local startup
+
+```mermaid
+sequenceDiagram
+    participant Dev as Developer tool
+    participant MF as Miniflare
+    participant Plugin as Artifacts plugin
+    participant Controller as ArtifactsController
+    participant Sidecar as Node Git sidecar
+    participant Workerd as workerd
+
+    Dev->>MF: Configure Artifacts binding
+    MF->>Plugin: getBindings()
+    Plugin->>Plugin: Select local and validate namespace
+    Plugin-->>MF: Bind to local namespace entrypoint
+    MF->>Plugin: getServices()
+    Plugin->>Plugin: Verify host Git is available on PATH
+    Plugin->>Controller: get(Git storage path)
+    Controller->>Sidecar: Start worker thread and private HTTP listener
+    Sidecar-->>Controller: Address and secret
+    Controller-->>Plugin: Sidecar endpoint and Git socket port
+    Plugin-->>MF: Binding Worker, DO storage, backend service, Git socket
+    MF->>Workerd: Start configured services and socket
+```
+
+`getBindings()` routes each local binding to a namespace-scoped service.
+`getServices()` creates the binding Worker, namespace Durable Object, persistent
+metadata disk, internal Git backend service, and Git socket. The controller
+reuses a sidecar per Git storage path and closes it when Miniflare is disposed.
+The socket used by Git clients and the sidecar's private HTTP listener are
+**different endpoints**.
+
+## Local binding RPC
+
+For example, `env.REPOS.create("demo")`:
+
+```mermaid
+sequenceDiagram
+    participant App as User Worker
+    participant Entry as Binding Worker
+    participant DO as Namespace DO
+    participant Backend as Backend service
+    participant Sidecar as Node Git sidecar
+    participant Disk as Local disk
+
+    App->>Entry: env.REPOS.create("demo")
+    Entry->>DO: RPC create("demo")
+    DO->>DO: Validate and serialize mutation
+    DO->>Backend: POST internal create request
+    Backend->>Sidecar: Inject private backend header
+    Sidecar->>Disk: git init --bare
+    Sidecar-->>DO: Git state (via backend service)
+    DO->>Disk: Persist metadata and token hash
+    DO-->>Entry: Repository info and initial token
+    Entry-->>App: Create result
+```
+
+The Durable Object stores repository metadata and token hashes on local disk;
+the sidecar stores the bare Git repositories separately. `git-sidecar.ts`
+handles HTTP routing and repository locks, while `git-client.ts` owns
+repository-scoped Git commands and result parsing. Read operations such as
+`readFile()` also use the binding Worker and native Git backend. Node callers
+reach the same binding through Miniflare's Node binding proxy. Namespace and
+repository names are hashed into short, namespace-scoped on-disk paths; this
+leaves room for Git packfiles and workerd SQLite files on platforms with path
+length limits. Older draft storage layouts are not migrated automatically:
+startup rejects a detected old layout rather than silently creating empty
+repositories. Back up any local repositories before resetting the Artifacts
+persistence directory. Do not delete the persistence directory without first
+checking whether it holds work you need.
+
+## Git clone and push
+
+```mermaid
+sequenceDiagram
+    participant Git as Git client
+    participant Socket as Git socket
+    participant Entry as Binding Worker
+    participant DO as Namespace DO
+    participant Backend as Backend service
+    participant Sidecar as Node Git sidecar
+    participant Native as Native Git
+
+    Git->>Socket: Git smart HTTP at /git/:namespace/:repo.git
+    Socket->>Entry: Route request
+    Entry->>DO: fetch(request)
+    DO->>DO: Check endpoint, repository, token, and scope
+    DO->>Backend: Forward request without Authorization header
+    Backend->>Sidecar: Inject private backend header
+    Sidecar->>Native: Stream via git http-backend
+    Native-->>Sidecar: Discovery or pack response
+    Sidecar-->>DO: Git response (via backend service)
+    opt Successful receive-pack push
+        DO->>Backend: Read updated refs
+        Backend->>Sidecar: Internal refs request
+        Sidecar-->>DO: Refs (via backend service)
+        DO->>DO: Persist updated push metadata
+    end
+    DO-->>Entry: Git response
+    Entry-->>Socket: Git response
+    Socket-->>Git: Git response
+```
+
+The binding Worker checks the supported Git endpoints and authenticates
+repository tokens before forwarding. The internal service injects the sidecar
+secret; the client's `Authorization` header is removed before that hop. For a
+successful push, the Durable Object refreshes repository metadata before
+returning the response. The sidecar uses native `git http-backend` for Git smart
+HTTP rather than reproducing Git's pack protocol.
+
+## Explicit remote binding
+
+When `dev.remote` is `true`, this binding uses the remote-bindings proxy instead
+of creating local Artifacts storage or a local Git sidecar for that namespace.
+
+```mermaid
+sequenceDiagram
+    participant App as User Worker
+    participant MF as Miniflare
+    participant Plugin as Artifacts plugin
+    participant Proxy as Remote-bindings proxy
+    participant CF as Cloudflare Artifacts
+
+    MF->>Plugin: getBindings() with remote: true
+    Plugin-->>MF: Bind to remote proxy service
+    MF->>Proxy: Start remote proxy Worker
+    App->>Proxy: env.REPOS method call
+    Proxy->>CF: Remote binding request
+    CF-->>Proxy: Result
+    Proxy-->>App: Result
+```
+
+A local binding does not automatically synchronize with production Artifacts.
+An explicit repository import may contact its specified HTTPS Git remote; that
+is distinct from enabling a remote Artifacts binding.