Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
# Changelog

## 0.4.1

- Search filenames by default, with explicit path and all-details scopes retained in saved searches, selection, and CSV exports.
- List directly matching folders and ZIP/7z/RAR names separately, with pagination and open/reveal controls. Archive contents remain opt-in.
- Match accented letter case in filenames and location names, while treating wildcard characters literally.
- Prevent fast duplicate checks and scans from overwriting completed backend progress with a running state.
- Use the same bounded, fingerprint-checked G-code import for selected files and the file picker, including large ordinary files.
- Retain imported material units and density; clear stale conversions for review and preserve manual quantities. Review legacy imports that lack conversion history.
- Preserve search locations and the previous inventory after a failed scan, and record the failed attempt separately.
- Include the complete scan/transfer/consent workflow in packaged desktop checks.
- Keep model results visible alongside location matches at the minimum window size.

Rescan after upgrading to index folder and archive names. Source models remain unchanged unless a reviewed transfer or new-copy export is confirmed.

## 0.4.0

- Compare matching STL, OBJ, and PLY triangle geometry across names and formats, separately from exact file duplicates.
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Search folders or drives, browse a cached gallery, keep multi-part projects toge

## Run it

Use Windows 10/11 x64. Build the release ZIP with the commands below, or download a published build when one is available under [Releases](https://github.com/the3dcoder/ModelScout/releases). Extract the complete ZIP and run **Model-Scout-0.4.0.exe**. No installer is required. The current build is unsigned.
Use Windows 10/11 x64. Build the release ZIP with the commands below, or download a published build when one is available under [Releases](https://github.com/the3dcoder/ModelScout/releases). Extract the complete ZIP and run **Model-Scout-0.4.1.exe**. No installer is required. The current build is unsigned.

The portable executable stores its catalog, thumbnails, profiles, and transfer logs in `%APPDATA%\Model Scout`. Back up that folder while the app is closed if you want to preserve your library metadata. Source models stay in their original folders until you confirm a move.

Expand All @@ -20,7 +20,7 @@ npm start
## Organize a library

1. Add search folders or drives. ZIP scanning is optional; 7z and RAR additionally require a local [7-Zip installation](https://www.7-zip.org/).
2. Search names, paths, tags, notes, and categories. Combine tags with all/any/exclude filters, mark favorites, and save searches.
2. Search filenames by default, without inheriting matches from parent folders or archives. Matching folder and ZIP/7z/RAR names appear separately; their contents are not included automatically. Choose **Paths** or **All file details** to search broader fields explicitly. Combine tags with all/any/exclude filters, mark favorites, and save searches.
3. Switch between the list and gallery. Thumbnails are generated for visible cards and cached; **Prepare this page** is an explicit background action.
4. Select related files and choose **Collection**. Keep creator, source link, license notes, and project notes with the set. One file can belong to several collections. Membership survives rescans; unavailable files are counted separately.
5. Run **Check duplicates**. Exact matches use SHA-256 after size filtering. Choose a keeper and review moving extras to a separate folder.
Expand All @@ -29,6 +29,8 @@ npm start

Category organization retains the original search-root folder, subfolders, and filenames. Collections are virtual groups, so adding a file to one does not move it. After an in-app move, scan the destination to see that file again with its collection, tags, favorites, notes, and saved cost estimate. The estimate retains its original fingerprint and may need review.

Search is insensitive to letter case, including accented names. Quoted phrases and multiple words are supported; `%` and `_` are literal characters. Folder/archive matches have their own pagination and open/reveal controls. File filters and bulk selection/export apply to model results. Archive names are indexed without unpacking them; searching inside archives remains opt-in. Rescan once after upgrading from 0.4.0 to populate folder/archive names. Existing saved searches use names by default; save them with a broader scope if needed.

Geometry comparison matches the same triangles at a fixed precision of 0.00001 model units. It ignores placement, vertex/triangle order, winding, normals, colors, and materials, while preserving scale and orientation. Units and intended use are not compared. Rotation, different triangulation, or precision differences may miss matches. These are review candidates, distinct from byte-identical duplicates; no file is automatically selected or removed. Processing is local and bounded to 64 MiB, 500,000 triangles, and 45 seconds per file. Completed results and failures are cached for unchanged files; cancel and resume, or explicitly retry failures. Groups are paged, with up to 80 files shown in each group.

**Include declared OBJ/glTF assets** adds supported local material, texture, and buffer references to the review plan. Shared support files are copied once per destination and their originals remain in place, even during a model move. Missing or unsafe references block the plan. Other project formats, extension-specific glTF references, and archive-contained projects may need manual review. External assets are preserved during transfer; previews do not fetch those textures or buffers.
Expand All @@ -39,6 +41,8 @@ Use filament or resin profiles with your own material prices, printer power and

Enter whole-job time and material from your slicer, or import supported text G-code comments. Export/import Model Scout profiles as versioned JSON. Estimates retain their inputs and file version. Omitted rates are flagged; example/default values are not market prices. Vendor printer profiles and proprietary resin job formats are not imported automatically. A mesh volume alone cannot establish realistic printing time or material use.

Ordinary G-code imports read bounded header/footer windows, consistently from the library and file picker, including files larger than the preview limit. Archive members retain the extraction limit. Imported material conversions retain source units and density. Changing the density or material unit clears a converted quantity for review; manually entered quantities stay unchanged when density changes. Older imported estimates without conversion history require a reviewed quantity before saving again.

## Model inspection and editing

STL tools report bounds, triangles, degenerate/duplicate faces, boundary and non-manifold edges, components, winding, and signed volume. Uniform scale, XYZ rotation, and removal of zero-area or duplicate triangles produce a preview and a **new STL file** with a verification receipt.
Expand Down
107 changes: 93 additions & 14 deletions app/catalog.cjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
const { DatabaseSync } = require("node:sqlite");
const crypto = require("node:crypto");
const fold = (text) =>
String(text || "")
.normalize("NFC")
.toLowerCase();
const termsOf = (search) =>
(
String(search || "")
.slice(0, 500)
.match(/"[^"]+"|\S+/g) || []
)
.slice(0, 8)
.map((raw) => fold(raw.replace(/^"|"$/g, "")));
const versionOf = (r) => `${r.size}:${r.mtime}:${r.ctime ?? ""}`;
const tagsOf = (value) =>
[
Expand All @@ -18,22 +30,29 @@ const present = (r) =>
class Catalog {
constructor(file) {
this.db = new DatabaseSync(file);
this.db.function(
"scout_contains",
{ deterministic: true },
(value, term) => +fold(value).includes(term),
);
this.db.exec("PRAGMA busy_timeout=5000; PRAGMA journal_mode=WAL");
const exists = this.db
.prepare("SELECT name FROM sqlite_master WHERE name='files'")
.get();
if (
exists &&
this.db.prepare("PRAGMA user_version").get().user_version < 4 &&
this.db.prepare("PRAGMA user_version").get().user_version < 5 &&
file !== ":memory:"
) {
this.backupPath = `${file}.before-v4-${Date.now()}.bak`;
this.backupPath = `${file}.before-v5-${Date.now()}.bak`;
this.db.prepare("VACUUM INTO ?").run(this.backupPath);
}
this.db
.exec(`CREATE TABLE IF NOT EXISTS files(id TEXT PRIMARY KEY,path TEXT,member TEXT,root TEXT,name TEXT,ext TEXT,size REAL,mtime REAL,ctime REAL,family TEXT,category TEXT,evidence TEXT,confidence TEXT,preview INTEGER,hash TEXT,analysis TEXT);
CREATE TABLE IF NOT EXISTS notes(id TEXT PRIMARY KEY,category TEXT,notes TEXT,tags TEXT NOT NULL DEFAULT '[]',favorite INTEGER NOT NULL DEFAULT 0);
CREATE TABLE IF NOT EXISTS settings(key TEXT PRIMARY KEY,value TEXT);
CREATE TABLE IF NOT EXISTS locations(path TEXT PRIMARY KEY,name TEXT NOT NULL,kind TEXT NOT NULL,root TEXT NOT NULL);
CREATE INDEX IF NOT EXISTS locations_name ON locations(name COLLATE NOCASE,path);
CREATE TABLE IF NOT EXISTS asset_cache(id TEXT PRIMARY KEY,size REAL,mtime REAL,ctime REAL,hash TEXT,analysis TEXT,thumbnail BLOB,thumbError TEXT);
CREATE TABLE IF NOT EXISTS collections(id TEXT PRIMARY KEY,name TEXT NOT NULL COLLATE NOCASE UNIQUE,creator TEXT NOT NULL DEFAULT '',license TEXT NOT NULL DEFAULT '',url TEXT NOT NULL DEFAULT '',notes TEXT NOT NULL DEFAULT '');
CREATE TABLE IF NOT EXISTS collection_members(collection_id TEXT NOT NULL,file_id TEXT NOT NULL,PRIMARY KEY(collection_id,file_id));
Expand Down Expand Up @@ -69,7 +88,7 @@ class Catalog {
});
}
this.db.exec(
"INSERT OR IGNORE INTO asset_cache(id,size,mtime,ctime,hash,analysis) SELECT id,size,mtime,ctime,hash,analysis FROM files; PRAGMA user_version=4",
"INSERT OR IGNORE INTO asset_cache(id,size,mtime,ctime,hash,analysis) SELECT id,size,mtime,ctime,hash,analysis FROM files; PRAGMA user_version=5",
);
this.insert = this.db.prepare(
"INSERT OR IGNORE INTO files(id,path,member,root,name,ext,size,mtime,ctime,family,category,evidence,confidence,preview,hash,analysis) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
Expand All @@ -94,7 +113,7 @@ class Catalog {
beginScan() {
this.flush();
this.db.exec(
"BEGIN; DROP TABLE IF EXISTS scan_backup; CREATE TABLE scan_backup AS SELECT * FROM files; DELETE FROM files; COMMIT",
"BEGIN; DROP TABLE IF EXISTS scan_backup; CREATE TABLE scan_backup AS SELECT * FROM files; DROP TABLE IF EXISTS locations_backup; CREATE TABLE locations_backup AS SELECT * FROM locations; DELETE FROM files; DELETE FROM locations; COMMIT",
);
}
finishScan(restore = false) {
Expand All @@ -110,6 +129,19 @@ class Catalog {
this.db.exec(
"DELETE FROM files; INSERT INTO files SELECT * FROM scan_backup",
);
if (
this.db
.prepare(
"SELECT name FROM sqlite_master WHERE name='locations_backup'",
)
.get()
) {
if (restore)
this.db.exec(
"DELETE FROM locations; INSERT INTO locations SELECT * FROM locations_backup",
);
this.db.exec("DROP TABLE locations_backup");
}
this.db.exec("DROP TABLE scan_backup; COMMIT");
return true;
} catch (e) {
Expand Down Expand Up @@ -176,18 +208,28 @@ class Catalog {
tags = [],
excludeTags = [],
tagMode = "all",
searchScope = "name",
} = options;
const terms =
String(search)
.slice(0, 500)
.match(/"[^"]+"|\S+/g) || [];
for (const raw of terms.slice(0, 8)) {
const term =
"%" + raw.replace(/^"|"$/g, "").replace(/[\\%_]/g, "\\$&") + "%";
for (const term of termsOf(search)) {
const fields =
searchScope === "all"
? [
"f.name",
"f.path",
"f.member",
"COALESCE(n.notes,'')",
"COALESCE(n.tags,'')",
"COALESCE(n.category,f.category)",
]
: searchScope === "path"
? ["f.path", "f.member"]
: ["f.name"];
clauses.push(
"(f.name LIKE ? ESCAPE '\\' OR f.path LIKE ? ESCAPE '\\' OR f.member LIKE ? ESCAPE '\\' OR COALESCE(n.notes,'') LIKE ? ESCAPE '\\' OR COALESCE(n.tags,'') LIKE ? ESCAPE '\\' OR COALESCE(n.category,f.category) LIKE ? ESCAPE '\\')",
"(" +
fields.map((field) => `scout_contains(${field}, ?)`).join(" OR ") +
")",
);
params.push(term, term, term, term, term, term);
params.push(...fields.map(() => term));
}
if (ext) {
clauses.push("f.ext=?");
Expand Down Expand Up @@ -271,7 +313,40 @@ class Catalog {
)
.all(...params, pageSize, page * pageSize)
.map(present);
return { rows, count, page, pageSize };
return { rows, count, page, pageSize, locations: this.locations(options) };
}
addLocation(file, kind, root) {
this.db
.prepare("INSERT OR IGNORE INTO locations VALUES(?,?,?,?)")
.run(file, require("node:path").basename(file) || file, kind, root);
}
location(file) {
return this.db
.prepare("SELECT * FROM locations WHERE path=?")
.get(String(file));
}
locations(options = {}) {
const terms = termsOf(options.search);
if (!terms.length) return { rows: [], count: 0, page: 0, pageSize: 20 };
const field = options.searchScope === "path" ? "path" : "name";
const where =
" WHERE " + terms.map(() => `scout_contains(${field}, ?)`).join(" AND ");
const params = terms;
const count = this.db
.prepare("SELECT COUNT(*) AS count FROM locations" + where)
.get(...params).count;
const page = Math.min(
Math.max(0, Math.floor(Number(options.locationPage) || 0)),
Math.max(0, Math.ceil(count / 20) - 1),
);
const rows = this.db
.prepare(
"SELECT * FROM locations" +
where +
" ORDER BY name COLLATE NOCASE,path LIMIT 20 OFFSET ?",
)
.all(...params, page * 20);
return { rows, count, page, pageSize: 20 };
}
stats() {
return {
Expand Down Expand Up @@ -460,6 +535,7 @@ class Catalog {
const query = {};
for (const k of [
"search",
"searchScope",
"ext",
"category",
"kind",
Expand All @@ -470,6 +546,9 @@ class Catalog {
])
query[k] = String(options?.[k] || "").slice(0, 500);
query.tags = tagsOf(options?.tags);
query.searchScope = ["name", "path", "all"].includes(query.searchScope)
? query.searchScope
: "name";
query.excludeTags = tagsOf(options?.excludeTags);
saved.push({ name, query });
}
Expand Down
112 changes: 72 additions & 40 deletions app/costs.cjs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
const fs = require("node:fs/promises");
const path = require("node:path");
const { dialog } = require("electron");
const { readRow } = require("./files.cjs");
const { readEstimatesText } = require("./gcode.cjs");
module.exports = function registerCosts({ handle, catalog, win, row }) {
// This pure calculation module is also used by the live calculator in the renderer.
const logic = () => import("../src/costing.mjs");
Expand Down Expand Up @@ -45,7 +45,7 @@ module.exports = function registerCosts({ handle, catalog, win, row }) {
const r = row(id);
if (!["gcode", "g", "gco"].includes(r.ext))
throw new Error("Select a text G-code file.");
text = (await readRow(r)).toString("utf8");
text = await readEstimatesText(r);
source = r.name;
} else {
const result = await dialog.showOpenDialog(win, {
Expand All @@ -55,48 +55,80 @@ module.exports = function registerCosts({ handle, catalog, win, row }) {
});
if (result.canceled) return null;
const file = result.filePaths[0],
stat = await fs.stat(file),
h = await fs.open(file, "r");
try {
const length = Math.min(stat.size, 256 * 1024),
head = Buffer.alloc(length),
tail = Buffer.alloc(length);
const first = await h.read(head, 0, length, 0);
const last =
stat.size > length
? await h.read(tail, 0, length, stat.size - length)
: { bytesRead: 0 };
text =
head.subarray(0, first.bytesRead).toString("utf8") +
"\n" +
tail.subarray(0, last.bytesRead).toString("utf8");
source = path.basename(file);
} finally {
await h.close();
}
stat = await fs.lstat(file);
if (![".gcode", ".g", ".gco"].includes(path.extname(file).toLowerCase()))
throw new Error("Select a text G-code file.");
text = await readEstimatesText({
path: file,
member: "",
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
});
source = path.basename(file);
}
const { parseGcodeEstimates } = await logic();
return { ...parseGcodeEstimates(text), source };
});
handle("saveCostEstimate", async (id, profile, inputs, source) => {
const r = row(id);
const { estimateCost, validateProfile } = await logic();
const result = estimateCost(profile, inputs);
const value = {
profile: validateProfile(profile),
inputs: {
hours: Number(inputs.hours),
amount: Number(inputs.amount),
parts: Number(inputs.parts),
},
source: String(source).slice(0, 300),
result,
date: new Date().toISOString(),
fileVersion: r.version,
};
catalog.setting("cost:" + id, value);
return value;
});
handle(
"saveCostEstimate",
async (id, profile, inputs, source, materialSource) => {
const r = row(id);
const { estimateCost, validateProfile } = await logic();
const result = estimateCost(profile, inputs);
let provenance = null;
if (materialSource) {
const { sourceUnit, unit } = materialSource;
const sourceAmount = Number(materialSource.sourceAmount),
density = Number(materialSource.density);
const converted = sourceUnit !== unit;
if (
!["g", "ml"].includes(sourceUnit) ||
unit !== profile.materialUnit ||
!Number.isFinite(sourceAmount) ||
sourceAmount < 0 ||
(converted &&
(!Number.isFinite(density) ||
density <= 0 ||
density !== Number(profile.density)))
)
throw new Error(
"Review the imported material quantity and density before saving.",
);
const expected = !converted
? sourceAmount
: unit === "g"
? sourceAmount * density
: sourceAmount / density;
if (Math.abs(Number(inputs.amount) - expected) > 0.000051)
throw new Error(
"Imported material quantity changed. Enter it manually or reimport.",
);
provenance = {
source: String(materialSource.source).slice(0, 300),
sourceUnit,
sourceAmount,
unit,
density: converted ? density : null,
};
}
const value = {
profile: validateProfile(profile),
inputs: {
hours: Number(inputs.hours),
amount: Number(inputs.amount),
parts: Number(inputs.parts),
},
source: String(source).slice(0, 300),
materialSource: provenance,
result,
date: new Date().toISOString(),
fileVersion: r.version,
};
catalog.setting("cost:" + id, value);
return value;
},
);
handle("costEstimate", (id) => {
row(id);
return catalog.setting("cost:" + id);
Expand Down
Loading