diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..d063cee --- /dev/null +++ b/.gitattributes @@ -0,0 +1,5 @@ +* text=auto eol=lf +*.ico binary +*.png binary +*.wasm binary +notices/** -whitespace diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..edbb1dc --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,30 @@ +# Changelog + +## 0.4.0 + +- Compare matching STL, OBJ, and PLY triangle geometry across names and formats, separately from exact file duplicates. +- Review candidates with paired previews, paged groups, and explicit selection that leaves a keeper in every group. +- Cancel and resume local comparisons using cached results; retry failed files on demand. +- Preserve exact-duplicate keeper choices across pages and wait for hashing to finish before review. +- Keep saved cost estimates after in-app moves and clear moved models from the inspector. +- Normalize Git checkout line endings so Windows CI can run formatting checks consistently. + +Geometry comparison preserves scale and orientation and ignores units and appearance. Rotated models, different triangulation, and precision differences may miss matches. No cleanup runs automatically. + +## 0.3.0 + +- Add collections with creator, source, license, and project notes. +- Include declared local OBJ/glTF assets in reviewed transfers. +- Strengthen cache, cancellation, metadata relocation, and inspector behavior. +- Prepare MIT source, runtime notices, importer source bundles, contributor documentation, and Windows checks. + +## 0.2.0 + +- Add static gallery thumbnails, favorites, tags, saved searches, and bulk metadata. +- Add filament/resin cost profiles and saved job estimates. +- Add basic STL diagnostics and verified export of edited copies. + +## 0.1.0 + +- Discover model and printing files, including opt-in archive scanning. +- Preview supported files, review SHA-256 duplicates, and copy or move files with verification and a transfer journal. diff --git a/README.md b/README.md index 882705a..b4beb75 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Search folders or drives, browse a cached gallery, keep multi-part projects toge ## Run it -Use Windows 10/11 x64. Extract the complete release ZIP and run **Model-Scout-0.3.0.exe**. No installer is required. The current build is unsigned. +Use Windows 10/11 x64. Build the release ZIP with the commands below, or download a published build when one is available under [Releases](https://github.com/the3dcoder/ModelScout/releases). Extract the complete ZIP and run **Model-Scout-0.4.0.exe**. No installer is required. The current build is unsigned. The portable executable stores its catalog, thumbnails, profiles, and transfer logs in `%APPDATA%\Model Scout`. Back up that folder while the app is closed if you want to preserve your library metadata. Source models stay in their original folders until you confirm a move. @@ -24,9 +24,12 @@ npm start 3. Switch between the list and gallery. Thumbnails are generated for visible cards and cached; **Prepare this page** is an explicit background action. 4. Select related files and choose **Collection**. Keep creator, source link, license notes, and project notes with the set. One file can belong to several collections. Membership survives rescans; unavailable files are counted separately. 5. Run **Check duplicates**. Exact matches use SHA-256 after size filtering. Choose a keeper and review moving extras to a separate folder. -6. Choose **Copy / move selected**, set a destination, and review the file-by-file plan. Existing files are never overwritten. Every copy is verified before a moved source is removed. +6. Use **Matching geometry** to compare selected STL, OBJ, and PLY files, or all supported files in the current catalog. Compare candidates in paired previews, keep at least one file per group, and explicitly select extras for transfer review. +7. Choose **Copy / move selected**, set a destination, and review the file-by-file plan. Existing files are never overwritten. Every copy is verified before a moved source is removed. -Category organization retains the original search-root folder, subfolders, and filenames. Collections are virtual groups, so adding a file to one does not move it. After an in-app move, scan the destination to see that file again with its collection, tags, favorites, and notes. +Category organization retains the original search-root folder, subfolders, and filenames. Collections are virtual groups, so adding a file to one does not move it. After an in-app move, scan the destination to see that file again with its collection, tags, favorites, notes, and saved cost estimate. The estimate retains its original fingerprint and may need review. + +Geometry comparison matches the same triangles at a fixed precision of 0.00001 model units. It ignores placement, vertex/triangle order, winding, normals, colors, and materials, while preserving scale and orientation. Units and intended use are not compared. Rotation, different triangulation, or precision differences may miss matches. These are review candidates, distinct from byte-identical duplicates; no file is automatically selected or removed. Processing is local and bounded to 64 MiB, 500,000 triangles, and 45 seconds per file. Completed results and failures are cached for unchanged files; cancel and resume, or explicitly retry failures. Groups are paged, with up to 80 files shown in each group. **Include declared OBJ/glTF assets** adds supported local material, texture, and buffer references to the review plan. Shared support files are copied once per destination and their originals remain in place, even during a model move. Missing or unsafe references block the plan. Other project formats, extension-specific glTF references, and archive-contained projects may need manual review. External assets are preserved during transfer; previews do not fetch those textures or buffers. diff --git a/app/catalog.cjs b/app/catalog.cjs index 5fb6443..13ab0f3 100644 --- a/app/catalog.cjs +++ b/app/catalog.cjs @@ -24,10 +24,10 @@ class Catalog { .get(); if ( exists && - this.db.prepare("PRAGMA user_version").get().user_version < 3 && + this.db.prepare("PRAGMA user_version").get().user_version < 4 && file !== ":memory:" ) { - this.backupPath = `${file}.before-v3-${Date.now()}.bak`; + this.backupPath = `${file}.before-v4-${Date.now()}.bak`; this.db.prepare("VACUUM INTO ?").run(this.backupPath); } this.db @@ -37,6 +37,8 @@ class Catalog { CREATE TABLE IF NOT EXISTS asset_cache(id TEXT PRIMARY KEY,size REAL,mtime REAL,ctime REAL,hash TEXT,analysis TEXT,thumbnail BLOB,thumbError TEXT); CREATE TABLE IF NOT EXISTS collections(id TEXT PRIMARY KEY,name TEXT NOT NULL COLLATE NOCASE UNIQUE,creator TEXT NOT NULL DEFAULT '',license TEXT NOT NULL DEFAULT '',url TEXT NOT NULL DEFAULT '',notes TEXT NOT NULL DEFAULT ''); CREATE TABLE IF NOT EXISTS collection_members(collection_id TEXT NOT NULL,file_id TEXT NOT NULL,PRIMARY KEY(collection_id,file_id)); + CREATE TABLE IF NOT EXISTS geometry_cache(id TEXT PRIMARY KEY,version TEXT NOT NULL,algorithm TEXT NOT NULL,size REAL,mtime REAL,ctime REAL,signature TEXT,details TEXT,error TEXT); + CREATE INDEX IF NOT EXISTS geometry_signature ON geometry_cache(signature); CREATE INDEX IF NOT EXISTS collection_files ON collection_members(file_id); CREATE INDEX IF NOT EXISTS files_size ON files(size); CREATE INDEX IF NOT EXISTS files_hash ON files(hash); @@ -67,7 +69,7 @@ class Catalog { }); } this.db.exec( - "INSERT OR IGNORE INTO asset_cache(id,size,mtime,ctime,hash,analysis) SELECT id,size,mtime,ctime,hash,analysis FROM files; PRAGMA user_version=3", + "INSERT OR IGNORE INTO asset_cache(id,size,mtime,ctime,hash,analysis) SELECT id,size,mtime,ctime,hash,analysis FROM files; PRAGMA user_version=4", ); this.insert = this.db.prepare( "INSERT OR IGNORE INTO files(id,path,member,root,name,ext,size,mtime,ctime,family,category,evidence,confidence,preview,hash,analysis) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)", @@ -318,6 +320,14 @@ class Catalog { .all(...params) .map((r) => r.id); } + queryGeometryRows() { + return this.db + .prepare( + "SELECT * FROM files WHERE ext IN ('stl','obj','ply') ORDER BY id", + ) + .all() + .map((r) => ({ ...r, version: versionOf(r) })); + } annotate(id, category, notes) { if (!this.get(id)) throw new Error("File not in catalog."); this.db @@ -431,6 +441,8 @@ class Catalog { .run(next, id); this.db.prepare("DELETE FROM collection_members WHERE file_id=?").run(id); this.db.prepare("DELETE FROM files WHERE id=?").run(id); + const estimate = this.setting("cost:" + id); + if (estimate) this.setting("cost:" + next, estimate); this.db.exec("COMMIT"); } catch (error) { this.db.exec("ROLLBACK"); diff --git a/app/geometry-worker.cjs b/app/geometry-worker.cjs new file mode 100644 index 0000000..849ad1a --- /dev/null +++ b/app/geometry-worker.cjs @@ -0,0 +1,114 @@ +const { parentPort, workerData } = require("node:worker_threads"); +const { createHash } = require("node:crypto"); +const { pathToFileURL } = require("node:url"); +const path = require("node:path"); + +(async () => { + const load = (name) => + import( + pathToFileURL( + path.join( + workerData.threeRoot, + "examples/jsm/loaders", + name + "Loader.js", + ), + ).href + ); + const data = workerData.buffer; + const bytes = data.buffer.slice( + data.byteOffset, + data.byteOffset + data.byteLength, + ); + const geometries = []; + if (workerData.ext === "obj") { + const { OBJLoader } = await load("OBJ"); + const object = new OBJLoader().parse(new TextDecoder().decode(bytes)); + object.traverse((node) => { + if (node.isMesh) geometries.push(node.geometry); + else if (node.isLine || node.isPoints) + throw new Error("Non-triangle OBJ data needs manual review."); + }); + } else { + const type = workerData.ext === "stl" ? "STL" : "PLY"; + const module = await load(type); + let geometry; + try { + geometry = new module[type + "Loader"]().parse(bytes); + } catch { + throw new Error( + `Could not parse ${type} triangle geometry. Check the file in its source application.`, + ); + } + if (workerData.ext === "ply" && !geometry.index) + throw new Error("PLY point clouds are not triangle meshes."); + geometries.push(geometry); + } + let triangles = 0; + const minimum = [Infinity, Infinity, Infinity], + maximum = [-Infinity, -Infinity, -Infinity]; + for (const geometry of geometries) { + const positions = geometry.getAttribute("position"); + const count = geometry.index?.count ?? positions?.count; + if (!positions || !count || count % 3) + throw new Error("No complete triangles found."); + triangles += count / 3; + if (triangles > 500000) + throw new Error( + "Geometry comparison supports up to 500,000 triangles per file.", + ); + for (let j = 0; j < count; j++) { + const id = geometry.index ? geometry.index.getX(j) : j; + if (!Number.isInteger(id) || id < 0 || id >= positions.count) + throw new Error("Invalid mesh index."); + const values = [ + positions.getX(id), + positions.getY(id), + positions.getZ(id), + ]; + if (!values.every(Number.isFinite)) + throw new Error("Invalid mesh coordinates."); + values.forEach((value, axis) => { + minimum[axis] = Math.min(minimum[axis], value); + maximum[axis] = Math.max(maximum[axis], value); + }); + } + } + if (!triangles) throw new Error("No triangle geometry found."); + const dimensions = minimum.map((value, i) => maximum[i] - value); + // Fixed model-unit precision preserves scale; min-corner translation removes placement differences. + // Triangle and vertex order, winding, normals, colors, and materials are deliberately excluded. + const canonical = [], + quantize = (value) => { + const number = Math.round(value * 100000); + if (!Number.isSafeInteger(number)) + throw new Error("Coordinates exceed comparison precision limits."); + return number; + }; + for (const geometry of geometries) { + const positions = geometry.getAttribute("position"), + count = geometry.index?.count ?? positions.count; + for (let j = 0; j < count; j += 3) { + const vertices = []; + for (let k = 0; k < 3; k++) { + const id = geometry.index ? geometry.index.getX(j + k) : j + k; + vertices.push( + [positions.getX(id), positions.getY(id), positions.getZ(id)] + .map((n, axis) => quantize(n - minimum[axis])) + .join(","), + ); + } + canonical.push(vertices.sort().join(";")); + } + } + canonical.sort(); + const hash = createHash("sha256").update("triangle-v1\0" + triangles + "\0"); + for (const face of canonical) hash.update(face + "\n"); + parentPort.postMessage({ + result: { + signature: hash.digest("hex"), + triangles, + dimensions, + precision: 0.00001, + }, + }); +})().catch((error) => parentPort.postMessage({ error: error.message })); diff --git a/app/geometry.cjs b/app/geometry.cjs new file mode 100644 index 0000000..a84d53b --- /dev/null +++ b/app/geometry.cjs @@ -0,0 +1,221 @@ +const { Worker } = require("node:worker_threads"); +const path = require("node:path"); +const { readRow, unchanged } = require("./files.cjs"); +const ALGORITHM = "triangle-v1"; +const LIMIT = 64 * 1024 * 1024; +function geometryTask(buffer, ext, signal) { + if (buffer.byteLength > LIMIT) + return Promise.reject( + new Error("Geometry comparison read limit is 64 MiB."), + ); + if (!["stl", "obj", "ply"].includes(ext)) + return Promise.reject(new Error("Supported formats: STL, OBJ, PLY.")); + if (signal?.aborted) return Promise.reject(new Error("Cancelled.")); + return new Promise((resolve, reject) => { + const worker = new Worker(path.join(__dirname, "geometry-worker.cjs"), { + workerData: { + buffer, + ext, + threeRoot: path + .dirname(path.dirname(require.resolve("three"))) + .replace("app.asar", "app.asar.unpacked"), + }, + resourceLimits: { maxOldGenerationSizeMb: 512 }, + }); + let settled = false; + const finish = (error, data) => { + if (settled) return; + settled = true; + clearTimeout(timer); + signal?.removeEventListener("abort", abort); + worker.terminate(); + error ? reject(error) : resolve(data); + }; + const abort = () => finish(new Error("Cancelled.")); + const timer = setTimeout( + () => finish(new Error("Geometry comparison exceeded 45 seconds.")), + 45000, + ); + signal?.addEventListener("abort", abort, { once: true }); + worker.once("message", (data) => + finish(data.error ? new Error(data.error) : null, data.result), + ); + worker.once("error", (error) => finish(error)); + worker.once("exit", () => finish(new Error("Geometry worker stopped."))); + }); +} +async function compareGeometry(catalog, options, signal, emit) { + const ids = options?.ids; + if (ids && (!Array.isArray(ids) || ids.length > 10000)) + throw new Error("Choose up to 10,000 files for comparison."); + const rows = ids + ? [...new Set(ids)].map((id) => { + const row = catalog.get(id); + if (!row) + throw new Error( + "A selected file is unavailable. Rescan and select again.", + ); + return row; + }) + : catalog.queryGeometryRows(); + const supported = rows.filter((row) => + ["stl", "obj", "ply"].includes(row.ext), + ); + const state = { + phase: "geometry", + running: true, + total: supported.length, + done: 0, + cached: 0, + analyzed: 0, + errorCount: 0, + errors: [], + started: Date.now(), + }; + let lastEmit = 0; + emit({ ...state, errors: [] }); + for (const row of supported) { + if (signal.aborted) break; + try { + await unchanged(row); + const cached = catalog.db + .prepare( + "SELECT * FROM geometry_cache WHERE id=? AND version=? AND algorithm=?", + ) + .get(row.id, row.version, ALGORITHM); + if (cached && !(options?.retry && cached.error)) { + state.cached++; + if (cached.error) throw new Error(cached.error); + } else { + if (row.size > LIMIT) + throw new Error("Geometry comparison read limit is 64 MiB."); + const result = await geometryTask(await readRow(row), row.ext, signal); + await unchanged(row); + if (catalog.get(row.id)?.version !== row.version) + throw new Error("File changed in the catalog. Rescan."); + catalog.db + .prepare( + "INSERT OR REPLACE INTO geometry_cache VALUES(?,?,?,?,?,?,?,?,?)", + ) + .run( + row.id, + row.version, + ALGORITHM, + row.size, + row.mtime, + row.ctime, + result.signature, + JSON.stringify(result), + null, + ); + state.analyzed++; + } + } catch (error) { + if (signal.aborted) break; + catalog.db + .prepare( + "INSERT OR REPLACE INTO geometry_cache VALUES(?,?,?,?,?,?,?,?,?)", + ) + .run( + row.id, + row.version, + ALGORITHM, + row.size, + row.mtime, + row.ctime, + null, + null, + error.message.slice(0, 500), + ); + state.errorCount++; + if (state.errors.length < 200) + state.errors.push({ path: row.path, message: error.message }); + } + state.done++; + if (Date.now() - lastEmit >= 250) { + emit({ ...state, errors: state.errors.slice() }); + lastEmit = Date.now(); + } + } + state.running = false; + state.phase = signal.aborted ? "cancelled" : "complete"; + catalog.setting("lastGeometryCheck", state); + emit(state); + return state; +} +const validJoin = + " FROM files f JOIN geometry_cache g ON g.id=f.id AND g.size=f.size AND g.mtime=f.mtime AND g.ctime IS f.ctime AND g.algorithm='triangle-v1' "; +function validateSelection(catalog, ids) { + if (!Array.isArray(ids) || !ids.length || ids.length > 10000) + throw new Error("Choose 1–10,000 candidates for review."); + const selected = [...new Set(ids)], + signatures = new Map(); + const lookup = catalog.db.prepare( + "SELECT g.signature" + + validJoin + + "WHERE f.id=? AND g.signature IS NOT NULL", + ); + const count = catalog.db.prepare( + "SELECT COUNT(*) AS count" + validJoin + "WHERE g.signature=?", + ); + for (const id of selected) { + const result = lookup.get(id); + if (!result) + throw new Error( + "A candidate changed or is unavailable. Rescan and compare again.", + ); + signatures.set( + result.signature, + (signatures.get(result.signature) || 0) + 1, + ); + } + for (const [signature, picked] of signatures) { + if (count.get(signature).count <= picked) + throw new Error( + "Keep at least one file in every matching geometry group before queuing extras.", + ); + } + return selected; +} +function groups(catalog, page = 0) { + const grouped = + "SELECT g.signature,COUNT(*) AS count" + + validJoin + + "WHERE g.signature IS NOT NULL GROUP BY g.signature HAVING COUNT(*)>1"; + const total = catalog.db + .prepare("SELECT COUNT(*) AS count FROM (" + grouped + ")") + .get().count; + page = Math.min( + Math.max(0, Math.floor(Number(page) || 0)), + Math.max(0, Math.ceil(total / 20) - 1), + ); + const result = catalog.db + .prepare(grouped + " ORDER BY count DESC,g.signature LIMIT 20 OFFSET ?") + .all(page * 20); + return { + total, + page, + groups: result.map((group) => ({ + ...group, + files: catalog.db + .prepare( + "SELECT f.id,g.details" + + validJoin + + "WHERE g.signature=? ORDER BY f.name,f.id LIMIT 80", + ) + .all(group.signature) + .map((item) => ({ + ...catalog.get(item.id), + geometry: JSON.parse(item.details), + })), + truncated: group.count > 80, + })), + }; +} +module.exports = { + geometryTask, + compareGeometry, + groups, + validateSelection, + ALGORITHM, +}; diff --git a/app/main.cjs b/app/main.cjs index a76fd17..6f12c85 100644 --- a/app/main.cjs +++ b/app/main.cjs @@ -153,6 +153,7 @@ else userData: app.getPath("userData"), hasKey: !!apiKey, savedSearches: catalog.savedSearches(), + lastGeometryCheck: catalog.setting("lastGeometryCheck"), view: catalog.setting("view") || "list", backupPath: catalog.backupPath || null, })); @@ -299,6 +300,31 @@ else }); return true; }); + const geometry = require("./geometry.cjs"); + handle("geometryCompare", (options = {}) => { + requireIdle(); + if ( + options.ids && + (!Array.isArray(options.ids) || options.ids.length > 10000) + ) + throw new Error("Choose up to 10,000 files."); + thumbnails.stop(); + job = new AbortController(); + geometry + .compareGeometry(catalog, options, job.signal, progress) + .catch((e) => + progress({ running: false, phase: "failed", message: e.message }), + ) + .finally(() => { + job = null; + }); + return true; + }); + handle("geometryGroups", (page) => geometry.groups(catalog, page)); + handle("geometrySelection", (ids) => { + requireIdle(); + return geometry.validateSelection(catalog, ids); + }); handle("duplicateGroups", (page = 0) => ({ total: catalog.stats().duplicateGroups, groups: catalog.db diff --git a/app/preload.cjs b/app/preload.cjs index 029a198..0c33e6e 100644 --- a/app/preload.cjs +++ b/app/preload.cjs @@ -18,6 +18,9 @@ const names = [ "saveAnalysis", "duplicates", "duplicateGroups", + "geometryCompare", + "geometryGroups", + "geometrySelection", "transferPlan", "transferExecute", "reveal", diff --git a/docs/DESIGN.md b/docs/DESIGN.md index f5ffcbf..6d4b65c 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -10,7 +10,9 @@ The scanner streams progress and commits in batches. An inventory snapshot survi Three.js supplies loaders, the MTL parser, previews, and STL export. One hidden renderer generates thumbnails for visible items. Assimp and OpenCascade conversion and STL diagnostics run in bounded workers. Previews block external resource fetching. -Collections link persistent file IDs to project metadata. They survive changes to scan scope and report unavailable members. Verified in-app moves remap personal metadata and collection memberships to the destination ID; that location becomes available when scanned. Files moved outside the app are not automatically reconciled. +Collections link persistent file IDs to project metadata. They survive changes to scan scope and report unavailable members. Verified in-app moves remap personal metadata, collection memberships, and saved estimates to the destination ID; that location becomes available when scanned. Estimates retain their source fingerprint for review. Files moved outside the app are not automatically reconciled. + +Geometry comparison reuses Three.js STL/OBJ/PLY loaders in a bounded worker. It subtracts the mesh minimum corner, quantizes coordinates at 0.00001 model units, sorts vertices within each face and faces within the mesh, and hashes that canonical triangle list. Scale and orientation are retained; appearance and units are excluded. This detects matching triangle exports, not arbitrary shape similarity. A separate fingerprint- and algorithm-bound cache prevents stale results from appearing after rescans. Main-process validation checks that selected candidates leave a keeper in every group, including off-page groups, before transfer review. ## File transfers diff --git a/docs/FEATURE_RESEARCH.md b/docs/FEATURE_RESEARCH.md index d27c3a5..a06a75f 100644 --- a/docs/FEATURE_RESEARCH.md +++ b/docs/FEATURE_RESEARCH.md @@ -1,6 +1,6 @@ # Feature research and roadmap -Research reviewed: 2026-09-27–28. Scope: local Windows model-library discovery, organization, printing estimates, and model inspection. +Research reviewed: 2026-09-27–28 and 2026-10-02. Scope: local Windows model-library discovery, organization, printing estimates, and model inspection. ## Evidence, not a market-wide ranking @@ -54,3 +54,13 @@ These larger features are not silently represented as implemented. Current bound Reviewed transfers now resolve declared local OBJ materials/textures and glTF buffers/images, preserve their relative paths, copy support files before models, and keep shared asset originals. The [glTF 2.0 specification](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html) defines buffer and image URI fields. Three.js MTLLoader supplies the material parser. Unsupported references stop automatic inclusion and require review; this is not a universal project dependency resolver. Reliability work binds preview reads to the restored file fingerprint after cancelled scans, limits transfer-progress payload size, preserves metadata after verified moves, and guards stale inspector updates. Public-source preparation adds MIT licensing for original code, complete runtime notices, importer source bundles, fixture-based Windows CI, and contributor documentation. + +## Delivered in 0.4.0 + +[Manyfold #3876](https://github.com/manyfold3d/manyfold/issues/3876) requests shared-file deduplication that preserves references from multiple projects. This is a qualitative signal that cleanup must account for project relationships, rather than a market-wide popularity ranking. Model Scout keeps explicit keeper and transfer review, including declared assets, rather than introducing symlink-based consolidation in this update. + +We evaluated [Trimesh comparison identifiers](https://trimesh.org/trimesh.comparison.html), whose documented design aims to tolerate rigid transforms and tessellation changes. Such identifiers remain candidates for a future broader similarity pass, which needs measured false-positive and false-negative rates. This update uses the existing [Three.js OBJ](https://threejs.org/docs/pages/OBJLoader.html) and [PLY](https://threejs.org/docs/pages/PLYLoader.html) parsers alongside STL, without adding a Python runtime or copying comparison code from Trimesh. + +Matching geometry review compares a canonical triangle list at fixed model-unit precision, ignoring placement and order but retaining scale and orientation. A match is a review candidate, not confirmation that units, appearance, or intended use agree. Paired previews, explicit candidate selection, cross-page keeper validation, cancellation, cached success/failure results, retry, and paged groups make the bounded comparison usable in a large catalog. This is an initial step toward the geometry-similarity priority above; rotated or differently tessellated models may still be missed. + +Regression fixes retain exact-duplicate keeper choices across pages, prevent reviewing a partial duplicate scan, preserve saved cost estimates after verified moves, clear moved files from the inspector, and normalize Git checkout line endings for hosted Windows formatting checks. The baseline source was pushed before this update began. diff --git a/docs/VERIFICATION.md b/docs/VERIFICATION.md index aaf0ddc..bec6523 100644 --- a/docs/VERIFICATION.md +++ b/docs/VERIFICATION.md @@ -7,10 +7,10 @@ Use Windows x64 and Node.js 24.15 or newer. Tests create disposable files under 1. Run npm ci and npm test for scanning, archives, duplicate hashes, exclusive verified transfers, cancellation recovery, metadata, collections, caches, - cost formulas, and STL edits. + cost formulas, STL edits, geometry matching, and keeper validation. 2. Run npm run build followed by npm run test:app for desktop interactions, visible preview completion, galleries, filters, cost and mesh dialogs, - collections, referenced assets, and importer loading. + collections, referenced assets, importer loading, and paired geometry review. 3. Run npm run format:check and npm audit --omit=dev. 4. Run npm run package and npm run test:packaged to verify the same workflows against the unpacked release executable, including actual WASM loading and @@ -19,14 +19,19 @@ Use Windows x64 and Node.js 24.15 or newer. Tests create disposable files under WASM hashes against pinned upstream revisions and packages sources/notices with the executable. Keep the generated SHA-256 file with the ZIP. -## Local acceptance, 0.3.0 +## Local acceptance, 0.4.0 -The Windows source suite has 20 passing core/feature tests. Desktop fixtures cover +The Windows source suite has 22 passing core/feature tests. Desktop fixtures cover scan/preview, reviewed duplicate moves, mocked cloud consent, static galleries, failed previews, favorites/tags, saved searches, FDM/resin estimates, profile import/export, mesh analysis/edit/export, collection lifecycle, referenced assets, OpenCascade previews, Assimp loading, and the license viewer. UI checks include -normal and minimum supported window sizes. +normal and minimum supported window sizes. Geometry fixtures compare STL/OBJ/PLY, +placement and winding changes, scale and shape differences, invalid files, +active-worker cancellation, cache reuse, changed-fingerprint invalidation, and +saved-estimate relocation. Desktop regression fixtures verify paired previews, +geometry keeper protection across 21 groups, and exact-duplicate keeper choices +across 51 groups. Source and packaged desktop suites each cover these workflows. The portable build is unsigned. Source tests and local packaged tests do not establish GitHub-hosted CI, code-signing, or every format/model combination. Cloud diff --git a/package-lock.json b/package-lock.json index 642dad7..4733a15 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "model-scout", - "version": "0.3.0", + "version": "0.4.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "model-scout", - "version": "0.3.0", + "version": "0.4.0", "license": "MIT", "dependencies": { "assimpjs": "^0.0.10", diff --git a/package.json b/package.json index c57b324..967f618 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "model-scout", - "version": "0.3.0", + "version": "0.4.0", "private": true, "description": "A local 3D file discovery and organization workbench", "author": "Big Bwain LLC", @@ -9,8 +9,8 @@ "scripts": { "build": "vite build", "start": "npm run build && electron .", - "test": "node --test tests/core.test.cjs tests/enhancements.test.cjs tests/organization.test.cjs", - "test:app": "node tests/app.test.cjs && node tests/library-app.test.cjs && node tests/production-app.test.cjs && node tests/organization-app.test.cjs && node tests/importers-app.test.cjs", + "test": "node --test tests/core.test.cjs tests/enhancements.test.cjs tests/organization.test.cjs tests/geometry.test.cjs", + "test:app": "node tests/app.test.cjs && node tests/library-app.test.cjs && node tests/production-app.test.cjs && node tests/organization-app.test.cjs && node tests/importers-app.test.cjs && node tests/geometry-app.test.cjs", "package": "npm run licenses && npm run build && electron-builder --win portable --x64", "format:check": "prettier --check app src tests scripts vite.config.mjs package.json", "test:packaged": "node scripts/test-packaged.cjs", @@ -21,7 +21,7 @@ "appId": "com.bigbwain.modelscout", "productName": "Model Scout", "directories": { - "output": "release/0.3.0" + "output": "release/0.4.0" }, "files": [ "app/**/*", @@ -41,7 +41,9 @@ "filter": [ "loaders/STLLoader.js", "exporters/STLExporter.js", - "loaders/MTLLoader.js" + "loaders/MTLLoader.js", + "loaders/OBJLoader.js", + "loaders/PLYLoader.js" ] } ], diff --git a/scripts/test-packaged.cjs b/scripts/test-packaged.cjs index ddaace7..abd848a 100644 --- a/scripts/test-packaged.cjs +++ b/scripts/test-packaged.cjs @@ -4,6 +4,7 @@ for (const file of [ "production-app", "organization-app", "importers-app", + "geometry-app", ]) { const result = spawnSync(process.execPath, [`tests/${file}.test.cjs`], { stdio: "inherit", diff --git a/src/GeometryPanel.jsx b/src/GeometryPanel.jsx new file mode 100644 index 0000000..946f857 --- /dev/null +++ b/src/GeometryPanel.jsx @@ -0,0 +1,274 @@ +import React, { useState, useEffect, useRef } from "react"; +import { Preview } from "./Preview"; + +export function GeometryPanel({ ids, onQueue, running }) { + const [scope, setScope] = useState(ids.length ? "selected" : "all"); + const [retry, setRetry] = useState(false), + [state, setState] = useState(null), + [message, setMessage] = useState(""); + const [result, setResult] = useState({ total: 0, page: 0, groups: [] }), + [chosen, setChosen] = useState(new Map()); + const [left, setLeft] = useState(null), + [right, setRight] = useState(null); + const request = useRef(0); + const load = async (page = 0) => { + const n = ++request.current; + try { + const value = await window.scout.geometryGroups(page); + if (n === request.current) setResult(value); + } catch (error) { + setMessage(error.message); + } + }; + useEffect(() => { + window.scout.info().then((info) => setState(info.lastGeometryCheck)); + load(); + return window.scout.onProgress((value) => { + if (value.phase === "geometry" || typeof value.analyzed === "number") { + setState(value); + if (!value.running) load(); + } + }); + }, []); + const toggle = (file, group) => + setChosen((current) => { + const next = new Map(current); + next.has(file.id) + ? next.delete(file.id) + : next.set(file.id, { + ...file, + signature: group.signature, + groupCount: group.count, + }); + return next; + }); + const start = async () => { + try { + setMessage(""); + setChosen(new Map()); + setLeft(null); + setRight(null); + await window.scout.geometryCompare({ + ids: scope === "selected" ? ids : undefined, + retry, + }); + } catch (error) { + setMessage(error.message); + } + }; + const selectedCounts = new Map(); + for (const file of chosen.values()) + selectedCounts.set( + file.signature, + (selectedCounts.get(file.signature) || 0) + 1, + ); + const missingKeeper = [...chosen.values()].some( + (file) => selectedCounts.get(file.signature) >= file.groupCount, + ); + const queue = async () => { + try { + const checked = await window.scout.geometrySelection([...chosen.keys()]); + onQueue(checked); + } catch (error) { + setMessage(error.message); + } + }; + return ( +
+ Find matching triangle geometry in STL, OBJ, and PLY exports. Placement, + vertex/triangle order, and winding are ignored; scale and orientation + are preserved. Precision is 0.00001 model units. Materials, colors, + normals, and units are not compared. Different triangulation or rotation + may miss a match. +
++ Local, one file at a time. Limit: 64 MiB / 500,000 triangles per file. + Cancelling keeps completed work; run again to reuse it. +
+ {state && ( ++ {state.running + ? "Comparing" + : state.phase === "cancelled" + ? "Cancelled" + : "Last comparison"} + : {state.done} / {state.total} · {state.cached} reused ·{" "} + {state.analyzed} analyzed · {state.errorCount} issues +
+ )} + {message &&{message}
} + {state?.errors?.length > 0 && ( ++ {error.path}: {error.message} +
+ ))} ++ Run a comparison to discover candidates. No match does not prove two + models are different. +
+ )} ++ Showing the first 80 files; review the rest in the library + before choosing a keeper. +
+ )} + {group.files.map((file) => ( ++ Bounds:{" "} + {file.geometry.dimensions + .map((n) => n.toFixed(3)) + .join(" × ")}{" "} + model units +
+ > + )} ++ Keep at least one file in every matching group before queuing extras. +
+ )} +