Skip to content

fix: skip named pipes, sockets and device nodes when scanning [patch] - #148

Merged
matt-edmondson merged 1 commit into
mainfrom
fix/skip-non-regular-files
Sep 29, 2026
Merged

matt-edmondson merged 1 commit into
mainfrom
fix/skip-non-regular-files

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Scan, DryRun, Stats and Deduplicate no longer hang when the tree contains a named pipe. The scan now keeps only regular files, so pipes, sockets and device nodes never reach the hasher.

Cause

On Unix, Directory.EnumerateFiles returns every entry that isn't a directory. .NET gives named pipes, sockets and device nodes the same Normal attribute as a regular file (checked on .NET 10), so WalkOptions can't filter them. FileHasher.ComputeHash then called File.OpenRead on the pipe, and that call blocks until a writer appears.

Fix

  • New FileType.IsRegularFile, called from FileScanner.ScanForFiles next to the existing ReparsePoint policy. Every verb shares that scan, so all four are fixed in one place.
  • The file type comes from SystemNative_LStat, the runtime's own shim that backs FileInfo on Unix. This avoids the struct stat layout problem raised on Hardlinked files are deduplicated like independent copies, risking silent data loss in snapshot/backup trees #132. The runtime defines FileStatus with fixed-width fields, and the shim normalizes the type bits to the same values on every Unix. Only Mode is read, at offset 4 after the 32-bit Flags field. The buffer is oversized so a field the runtime appends later can't overrun it.
  • On Windows it always returns true. If the stat call fails, it also returns true, so hashing reports the problem the way it did before.
  • The project now sets AllowUnsafeBlocks, because the LibraryImport source generator needs it.

Tests

  • ScanLeavesOutANamedPipe: a mkfifo inside a TempTree is left out, and an empty regular file next to it is still found.
  • ScanAndHashFinishWhenTheTreeHoldsANamedPipe: scan and hash finish within 30 s, so a hang fails the test instead of stalling the run.
  • Both tests are marked inconclusive on Windows, which has no file-system pipes.
  • Verified by reversal: without the FileScanner/FileType change, both tests fail. The second one hits the 30 s timeout. With the change, the full suite passes: 55 passed, plus 2 privilege-dependent tests that were already inconclusive because the environment runs as root.
  • Manual check: Scan over a directory holding two copies, a FIFO and a Unix socket now finishes and reports the one duplicate group. On main the same run stops after Hashed: a -> ....
  • Verified on Linux x64 only. CI covers Windows and macOS.

Risk

SystemNative_LStat is a runtime export, not a documented public API. It has shipped since .NET Core 1.0, and Flags and Mode have always been its first two fields. If the export ever disappeared, the call would throw at scan time and the new tests would fail on every Unix leg.

Relates to #132 (hardlinks). The same shim also returns Dev and Ino, which may be a way past that issue's interop blocker. Deciding that is left to #132.

Fixes #136

🤖 Generated with Claude Code

https://claude.ai/code/session_01PNmrp6FP3tBU47owLsiovc


Generated by Claude Code

On Unix, Directory.EnumerateFiles yields every entry that is not a
directory, and .NET marks named pipes, sockets and device nodes with the
same Normal attribute as a regular file. The hasher then opened a named
pipe for reading, which blocks until a writer appears, so a single pipe
anywhere in the tree hung Scan, DryRun, Stats and Deduplicate.

ScanForFiles now keeps only regular files. The file type is read through
the runtime's SystemNative_LStat shim, whose FileStatus layout the runtime
defines with fixed-width fields and normalized type bits on every Unix,
so no per-platform struct stat layout is involved.

Fixes #136

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PNmrp6FP3tBU47owLsiovc
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scan, DryRun, Stats and Deduplicate hang forever when the tree contains a named pipe (FIFO)

2 participants