On Windows, OneDrive and other cloud-synced folders are silently skipped because the scan skips every reparse point, not just symlinks and junctions

[matt-edmondson]

What's wrong

FileDeduplicator/FileScanner.cs:38-43 sets AttributesToSkip = FileAttributes.ReparsePoint. The remarks above it (lines 23-26) say the intent is to avoid descending into directory symlinks that point back at an ancestor, which is the fix for #125.

On Windows, FILE_ATTRIBUTE_REPARSE_POINT is not limited to links. It is also set on:

• OneDrive / Files-On-Demand (Cloud Files API) files and folders, which carry IO_REPARSE_TAG_CLOUD_* tags. This includes files already hydrated locally, which is why Get-ChildItem lists them with mode l.
• Files on Windows Server Data Deduplication volumes (IO_REPARSE_TAG_DEDUP).
• Some other filter-driver placeholders, such as HSM and backup/archive stubs.

FileSystemEnumerator applies AttributesToSkip before it decides whether to recurse. A skipped cloud folder is therefore never entered.

Failure scenario

On a Windows machine with OneDrive:

1. Run Scan -p C:\Users\me\OneDrive\Pictures.
2. The tool reports far fewer files than the folder holds, possibly zero.
3. It prints no warning, so the result looks like a clean scan with no duplicates.

A OneDrive photo library is one of the most likely targets for this tool, and the README promises that it "Recursively scans all files".

How this was verified

This comes from tracing the code against documented Windows and .NET behaviour. It has not been reproduced: this review ran on Linux, where only symlinks carry the attribute.

Suggested fix

Skip only reparse points that are real links, meaning name surrogates such as symlinks and junctions:

• Replace the plain EnumerationOptions walk with a FileSystemEnumerable<string>.
• In both ShouldIncludePredicate and ShouldRecursePredicate, reject an entry only when it has FileAttributes.ReparsePoint and entry.ToFileSystemInfo().LinkTarget != null. LinkTarget is null for reparse points that are not links, such as cloud placeholders.

Hashing a placeholder that is not hydrated forces a download. Either accept that, or skip entries that have FileAttributes.Offline or FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS (0x400000) and report how many were skipped.

Acceptance criteria

• Symlink and junction cycles are still not followed, and the Scan crashes on any inaccessible directory and never terminates on a symlink cycle #125 tests still pass.
• Regular files under a reparse-point directory that is not a link are scanned.
• The include and recurse predicate logic is unit-tested in isolation, since a real cloud-files test is impractical.

[matt-edmondson]

Triage

• Category: Bug
• Priority: High. On Windows, OneDrive and other cloud-synced folders, which are among the likeliest things to deduplicate, are skipped with no warning. The result looks like a clean scan. This comes from tracing the code and has not been reproduced on Windows yet.
• Area / suggested assignee: FileScanner enumeration (AttributesToSkip = ReparsePoint, introduced by the Scan crashes on any inaccessible directory and never terminates on a symlink cycle #125 fix). Owner: @matt-edmondson
• Duplicates / in progress: not a duplicate. It is related to Scan crashes on any inaccessible directory and never terminates on a symlink cycle #125, whose symlink-cycle protection must be kept. No open PR covers it.
• Next step: switch to a FileSystemEnumerable and skip only reparse points that are real links (LinkTarget != null). Decide whether to skip and count offline or recall-on-access placeholders rather than hydrate them. Unit-test the predicates, and if possible reproduce once on a Windows OneDrive folder.

---

Generated by Claude Code