Add backwards-compatible BIP-329 label import and export

[schnuartz-ai]

Summary

Adds backwards-compatible BIP-329 label interoperability as a separate layer around Specter's existing address-label and frozen-UTXO state.

• adds deterministic UTF-8 JSONL export for explicit address labels and current wallet outputs
• preserves frozen outputs with spendable:false, including unlabeled frozen UTXOs
• imports BIP-329 addr labels through Specter's existing address-label store
• imports output spendable state through an explicit idempotent Core/Specter reconciliation operation
• serializes frozen-UTXO and pending-PSBT Core-lock mutations plus wallet JSON snapshots per wallet, and reuses the safe reconciliation operation in the existing freeze UI
• protects pending-PSBT inputs and Core locks without a matching persisted Specter freeze marker
• repairs a lost Bitcoin Core lock when Specter's persisted frozen_utxo marker still owns the freeze
• separates the verified wallet JSON write from callbacks and balance refreshes
• reports output.label as unsupported instead of lossily converting it into an address label
• refreshes the UTXO set before state-dependent BIP-329 import and export, and holds the wallet UTXO-state lock across export refresh and serialization
• reconstructs locked Bitcoin outputs from the actual raw outpoint and locked confidential Liquid outputs from wallet-aware unblinded data
• detects the string type/ref BIP-329 envelope, including unknown future types, without diverting legacy JSON that merely contains a type key
• applies BIP-329 size limits only after detecting BIP-329, preserving legacy import behavior
• does not count an already-identical address label as a new import
• keeps the Specter JSON backup format and Wallet.export_labels() unchanged
• retains Specter JSON, Electrum JSON, and Specter CSV imports, plus the canonical wallets_endpoint.settings URL endpoint
• introduces no database or wallet-file migration

Frozen-state and pending-PSBT lock safety

BIP-329 imports and the existing UI freeze action both use set_frozen_state(), which reads Core's current lock set and Specter's persisted marker together. It is idempotent, refuses pending-PSBT inputs and Core locks without a matching local marker, repairs missing non-persistent Core locks, and requires a successful boolean lockunspent response before changing local state. A wallet-specific reentrant lock covers pending-PSBT save/delete, the pending-input and Core-state reads, Core RPCs, RAM mutations, and wallet writes. The UI action also uses this reconciliation operation: it cannot unfreeze pending-PSBT inputs, and failed Core RPCs leave the local marker unchanged. Every wallet JSON snapshot/write takes the same lock, preventing an older concurrent save from overwriting newer UTXO-ownership state. Deleting a pending PSBT unlocks only inputs that are not still protected by a Specter freeze or another pending PSBT.

The persistence sequence has an explicit commit boundary:

Core RPC
   |
RAM update
   |
verified wallet JSON write
   |
PERSISTED
   |
storage callbacks and balance refresh

Only failure of the verified wallet JSON write restores the prior RAM/file snapshot and independently compensates a successful Core RPC. Failure of either rollback emits a separate privacy-preserving critical log.

Storage callbacks and update_balance() run after the commit. If either fails, the Core, RAM, and wallet JSON remain in the new consistent state and the import remains reported as updated; a post-commit side-effect failure is logged without label or outpoint metadata.

Why export and import are intentionally asymmetric

Specter stores explicit labels only on addresses, but uses an address label as the effective label displayed for that address's UTXOs. BIP-329 can represent address and output labels independently.

Sparrow's current BIP-329 importer also handles addr and output records independently: importing an addr record labels the address node, but it does not propagate that label to existing outputs. An addr-only export would therefore lose the UTXO-label semantics Specter users currently see and use for coin control when migrating to Sparrow.

For that reason, this adapter deliberately materializes each explicit Specter address label on every current known output for that address:

Specter address label
        |
        +--> BIP-329 addr record
        |
        +--> BIP-329 output record for each current known UTXO

These output records are a derived interoperability representation; they do not imply that Specter stores independent per-output labels.

The reverse conversion is unsafe. Another wallet may assign different labels to outputs on a reused address, including spent historical outputs. Collapsing those distinctions into one Specter address label would destroy information and could change the apparent labels of unrelated transactions. Therefore output.label is exported, but arbitrary imported output.label values are reported as unsupported rather than written to Specter's address store.

Concrete repeated-output case: even if ten current UTXOs on the same address all contain output.label = "Alice", Specter still does not infer the address label Alice. Agreement among the current UTXOs cannot prove that spent historical outputs on a reused address had the same meaning. An explicit addr record is required to update the Specter address label. Any spendable value on those ten records is still processed independently for each known outpoint.

There are two architectural paths:

1. This PR's backwards-compatible adapter: retain Specter's existing canonical address-label model and project its current semantics into BIP-329 for interoperability.
2. A future full label-model redesign: store address, transaction, and output labels independently and make BIP-329 concepts canonical inside Specter.

This PR intentionally chooses the first path and does not close the broader architecture problem tracked in upstream issue #2018.

Label conflict behavior

Malformed records are validated atomically. Conflicting duplicate address or spendable records are skipped rather than resolved by file order. Unknown wallet references do not create state. Unknown BIP-329 types are ignored for forward compatibility. Output labels from imported files are reported as unsupported and are never silently collapsed into an address label. Optional origin values are type-checked as strings but are not parsed or used to select a wallet: supported records must already resolve to the selected wallet, and different labels for the same reference remain conflicting even when their origins differ.

Known limitations

Bitcoin Core does not expose an owner for lockunspent locks. A Core lock without a matching Specter marker is safely rejected. If a persisted Specter marker exists, its original Core lock disappears, and another process later locks the same outpoint, the locks are indistinguishable; the importer treats the persisted marker as ownership. Pending PSBT inputs are protected independently of this marker.

Frozen-state imports currently reconcile and persist each distinct outpoint separately. A multi-outpoint batch would reduce RPC reads and file writes, but requires transaction and rollback semantics across several Core operations and is intentionally left for a separate change.

BIP-329 suggests, but does not require, a 255-character label limit. This adapter does not silently truncate BIP-329 labels while Specter's existing label model and legacy importers accept longer values. The existing 1 MiB per-line and 10 MiB document limits bound BIP-329 parser input.

Compatibility

The implementation follows the current BIP-329 UTF-8 JSON Lines format and was checked against Sparrow's current WalletLabels implementation. Existing Specter backup/export structures are unchanged.

Sparrow source inspected: WalletLabels.java at commit 3dc99b6.

Validation

• 59 focused unit/endpoint tests passed on the current head, including Bitcoin locked-output reconstruction and confidential Liquid locked-UTXO/detail/decoder regression tests
• a real temporary-file test verifies that a verified JSON write followed by an update_balance() exception leaves Core, RAM, and disk in the new committed state and reports one update with zero failed records
• a separate post-commit callback test verifies the same commit semantics and confirms that the balance refresh still runs
• wallet-write, persisted-state rollback, Core rollback, RPC exception, boolean-false, pending-PSBT, stale-lock, deterministic freeze/unfreeze, pending-save/freeze, pending-delete/unfreeze, concurrent general-save snapshot ordering, shared-lock ownership, deterministic export/freeze snapshot serialization, legacy-toggle locking and ownership/RPC-failure safety, settings-endpoint compatibility, origin conflict/type validation, Unicode, escaping, conflict, idempotence, unknown-future-type, and legacy detection paths are covered
• Python compilation, Black, and git diff --check passed
• the updated real Bitcoin Core regtest could not be executed in the current Windows environment because bitcoind is unavailable; the failure occurred during fixture setup
• Upstream PR checks passed on the current UI freeze and settings-compatibility fix (be2ac91f) for test, cypress, extension-smoketest, black, and the Linux smoke-test; the Netlify docs preview was canceled without a docs change

The full repository suite was not run locally; upstream test jobs passed on the current UI freeze and settings-compatibility commit. The Liquid regression tests use a synthetic value commitment and mocked wallet-aware RPC responses; no live Elements regtest was available locally. A manual Testnet3 Sparrow ↔ Specter roundtrip with a real frozen UTXO and screenshots is documented in the PR comments.

[netlify]

✅ Deploy Preview for specter-desktop-docs canceled.

Name	Link
🔨 Latest commit	be2ac91
🔍 Latest deploy log	https://app.netlify.com/projects/specter-desktop-docs/deploys/6aa68b2b39f941000735cc0d

[al-munazzim]

Thanks for the detailed BIP-329 interoperability work. A first maintainer pass is blocked by the current CI state: the Tests / cypress job is failing on the latest head (e4fe4f0).

The failing run is https://github.com/cryptoadvance/specter-desktop/actions/runs/34609800719. GitHub reports the failed Cypress spec as spec_ghost_machine.js / Ghost machine -- Check wallet settings; the uploaded cypress-results XML only contains the passing spec_plugins.js tests, so please check the Cypress screenshots artifact for the actual failure and push a fix or rerun if it is clearly flaky.

Once the PR is green, this will need a careful review because it changes wallet freeze/lock persistence and import/export semantics in a fairly large surface area.

[schnuartz-ai]

Manual Testnet3 interoperability check on PR commit ba50e23b; Sparrow 2.5.4 and an isolated local Specter/Spectrum instance):

• Used the same real, unspent 38,759-sat testnet output in both wallets (4dbef424…:0). No new transaction or funds movement was needed.
• Sparrow → Specter: Imported Sparrow's unmodified 56-record BIP-329 JSONL. The address label a and spendable:false were represented in Specter. To test an actual transition, I first imported spendable:true (Specter's frozen marker cleared), then re-imported Sparrow's file (spendable:false; marker restored and the UTXO showed frozen). Specter reported one unsupported output.label (x), as designed: it does not collapse an independent output label into an address label. No malformed, conflicting, or failed records were reported.
• Specter → Sparrow: Specter's separate JSONL export contained two addr records and one output record for that actual outpoint, with derived label a and spendable:false. Importing it through Sparrow's Labels → Import File changed Sparrow's output label from x to a; the 38,759-sat output remained frozen (Unfreeze UTXO in Sparrow's context menu).
• The live test exposed a pre-existing locked-UTXO reconstruction bug: Spectrum's gettransaction().details can contain a negative SEND accounting entry with the same vout as a CHANGE output. The new commit now derives the locked output's address and amount from the actual raw transaction output. Two regression tests cover this; all 50 focused BIP-329 tests, Black, and git diff --check pass locally. Full upstream CI passed on this exact head: test, cypress, extension-smoketest, black, and the Linux build/smoke-test.

Sparrow before import (output.label = x, frozen):

Specter after Sparrow import (same output, address-derived a, frozen):

Sparrow after Specter's JSONL import (output.label = a, still frozen):

These are app-only screenshots from the local test. The files contain testnet wallet metadata; no seed, private key, or other desktop content is shown. The adapter does not solve the independent transaction/output-label model discussed in #2018.

[schnuartz-ai]

Follow-up: fixed the Liquid/Elements locked-confidential-UTXO regression in 2d7fee2c.

LWallet inherits check_utxo(), but a confidential LTransactionOutput.value is a commitment, not an integer satoshi amount. The earlier Bitcoin fix therefore could raise while refreshing a locked Liquid UTXO. The shared method now calls a chain-specific resolver. Bitcoin still reads the actual raw outpoint (avoiding Spectrum's SEND/CHANGE accounting collision); Liquid takes an unblinded gettransaction().details entry only when its vout and decoded address script match the actual output, and otherwise uses Specter's existing Liquid-aware decoderawtransaction() fallback. It rejects a missing/unblinded-zero value rather than manufacturing a UTXO amount.

Two new automated tests use a synthetic confidential value commitment: one covers a conflicting SEND detail preceding the valid wallet detail and the other covers a missing change detail, decoder unblinding and confidential-address lookup. The 52 focused tests pass locally. On this exact commit, upstream test, cypress, extension-smoketest, black and Linux smoke-test checks passed. A live Elements regtest was not available locally; the separate Netlify docs deploy preview failed even though no docs files changed.

The earlier manual Sparrow ↔ Specter Testnet3 screenshots remain evidence for the Bitcoin path on the parent commit; they are not presented as a live Liquid test.

[schnuartz-ai]

Review follow-up on current head 54d3ca0: the Liquid locked-confidential-output regression was fixed in 2d7fee2 with wallet-aware unblinding and two regression tests. This commit closes the export snapshot race: check_utxo() and the complete BIP-329 export refresh/serialization now share the per-wallet UTXO-state RLock used by freeze and pending-PSBT mutations. A deterministic export-vs-freeze test verifies that a concurrent freeze cannot enter its Core-state read until the export completes. All 53 focused tests pass locally; the upstream test, cypress, black, extension-smoketest, and Linux smoke-test jobs pass on 54d3ca0. The existing UI toggle's error handling and moving post-commit callbacks out of the lock remain separate follow-up concerns; this PR has not changed their semantics. No merge has been performed.

[schnuartz-ai]

Review follow-up on head be2ac91: both findings were confirmed and fixed. The existing freeze UI now delegates to the same set_frozen_state() reconciliation used by BIP-329, so a pending-PSBT input cannot be unfrozen through the UI and an exception or False from lockunspent cannot change Specter's frozen marker. The history route flashes a safe error instead of returning a server error. The canonical wallet settings route again owns wallets_endpoint.settings; GET subaction aliases have distinct endpoints, and the BIP-329 import redirect uses the canonical endpoint. Six regression cases were added (pending-PSBT ownership, four freeze/thaw RPC-failure combinations, and URL generation). All 59 focused tests pass locally; test, cypress, black, extension-smoketest, and Linux smoke-test pass on this exact commit. The PR has not been merged. Live Elements regtest and moving post-commit callbacks out of the wallet lock remain separate follow-up concerns.