Skip to content

Preserve trailing whitespace in csvgrep matchfile patterns - #1367

Open
sakura168899-debug wants to merge 1 commit into
wireservice:masterfrom
sakura168899-debug:fix/csvgrep-matchfile-whitespace
Open

sakura168899-debug wants to merge 1 commit into
wireservice:masterfrom
sakura168899-debug:fix/csvgrep-matchfile-whitespace

Conversation

@sakura168899-debug

Copy link
Copy Markdown

Problem

csvgrep -f promises exact cell matches after removing line separators from each matchfile entry. However, line.rstrip() also removes data-significant spaces, tabs and Unicode whitespace. A pattern "value " therefore selects a row containing "value" and misses the actual exact match. Inverted matching selects the opposite wrong set.

Minimal reproduction (the space before \n in the matchfile is intentional):

from pathlib import Path

Path("input.csv").write_text("key,label\nvalue,trimmed\nvalue ,exact\n", encoding="utf-8")
Path("matches.txt").write_text("value \n", encoding="utf-8")
csvgrep -c key -f matches.txt input.csv

Expected: the exact row, whose key is "value ". Before this change, the trimmed row is returned instead.

Change

Restrict stripping to \r and \n. This changes one line in the matchfile reader, without changing regex/string matchers, case sensitivity or row-filter logic.

Regression tests cover trailing spaces, tabs, NBSP, em-space, whitespace-only patterns, LF/CRLF/CR/no final newline, multiple patterns, inverted matching and any/all-column modes. This does not implement the separate ignore-case proposal in #1356, whose current diff retains the original stripping behavior.

Validation

  • Unchanged baseline: 355 tests and 31 subtests pass. Before the runtime fix, the new coverage produces 24 data-assertion failures (22 subtests and two ordinary failures).
  • Publication recheck: 360 tests and 57 subtests pass; flake8, isort and git diff --check pass.
  • Normal python -m build --no-isolation sdist-to-wheel build, followed by a separate pip --target installation.
  • The installed wheel passes 16 new real-CLI SQLite/CSV contracts using fresh invoice references, including trailing spaces, tabs, narrow NBSP, whitespace-only and numeric-looking keys. All four line endings, inverse selection and original row numbering are checked against complete expected output.

Validation is local on Windows / CPython 3.12.14. The other OS/Python CI matrix and check-manifest have not been run locally. Existing warnings are unchanged.

AI assistance

This patch and its tests were prepared with Codex (AI assistance). I am an external contributor, not a project maintainer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant