Skip to content

Deduplicate deletes all but one empty file (__init__.py, .gitkeep, py.typed) while reclaiming 0 bytes #151

Description

@matt-edmondson

What's wrong

All zero-length files have the same SHA-256, so FindDuplicates (FileDeduplicator/Deduplicator.cs:31-34) puts every empty file in the scanned tree into one group. DeleteDuplicates (Deduplicator.cs:39-99) then keeps the copy with the shortest filename and deletes the rest.

Deleting them frees nothing. But an empty file usually matters because it exists: Python __init__.py, .gitkeep placeholders, py.typed markers, sentinel and lock files.

Failure scenario

  1. The tree contains a/__init__.py, b/__init__.py and logs/.gitkeep, all empty.
  2. Run Deduplicate -p ~/src. It reports one group of 3 with "Space to reclaim: 0 B".
  3. Answer y. .gitkeep is kept because it has the shortest name, and both __init__.py files are deleted. Both Python packages stop importing.

That is irreversible damage for zero benefit.

DeduplicatorTests.EmptyFilesAreDuplicatesOfEachOther asserts that empty files form a group, so reporting them is deliberate. Deleting them is the problem.

Suggested fix / acceptance criteria

  • Leave zero-length groups out of DeleteDuplicates and the deletion plan. Scan and Stats can still show them, for example as "N empty files (not deduplicated)".
  • Optionally, add an opt-in --include-empty switch for anyone who really wants empty files collapsed.
  • Test: Deduplicate on two empty files plus one real duplicate pair deletes only the real duplicate and leaves both empty files in place.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions