Skip to content

Weird output for long string equality assert diffs #7127

Closed
Closed
Weird output for long string equality assert diffs#7127
Labels
topic: reportingrelated to terminal output and user-facing messages and errorstopic: rewriterelated to the assertion rewrite mechanism

Description

@altendky
altendky
opened on Apr 28, 2020

Good:

E       AssertionError: assert 'a\na\na\na\na\na\na\na

Bad:

E       AssertionError: assert ('a\n'\n 'a\n'\n 'a\n'\n

Difference: The strings being compared are one character longer in the bad case.

This came up while asserting equality of GitHub Actions config files (strs) in PyCharm and noting that it wasn't showing the handy <Click to see difference> link (https://youtrack.jetbrains.com/issue/PY-42076). To be clear, the captures below were taken from a shell so as to avoid PyCharm messing things up. The PyCharm issue happens at larger lengths than this does.

This issue was referred to in #6757 but that ticket seems focused on other aspects of the output.

altendky@p1:~/repos/ciborg$ uname -a
Linux p1 5.3.13-altendky-iss510-b2d13b5a459a4dd0c571812011d1588fa77dfea7 #1 SMP Mon Dec 2 13:22:03 EST 2019 x86_64 x86_64 x86_64 GNU/Linux
altendky@p1:~/repos/ciborg$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 19.10
Release:        19.10
Codename:       eoan
altendky@p1:~/repos/ciborg$ virtualenv -p python3.8 venv.test
created virtual environment CPython3.8.2.final.0-64 in 115ms
  creator CPython3Posix(dest=/home/altendky/repos/ciborg/venv.test, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/altendky/.local/share/virtualenv/seed-app-data/v1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
altendky@p1:~/repos/ciborg$ venv.test/bin/pip install git+https://github.com/pytest-dev/pytest#egg=pytest
Collecting pytest
  Cloning https://github.com/pytest-dev/pytest to /tmp/pip-install-ozh3cks8/pytest
  Running command git clone -q https://github.com/pytest-dev/pytest /tmp/pip-install-ozh3cks8/pytest
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting attrs>=17.4.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting py>=1.5.0
  Using cached py-1.8.1-py2.py3-none-any.whl (83 kB)
Collecting packaging
  Using cached packaging-20.3-py2.py3-none-any.whl (37 kB)
Collecting pluggy<1.0,>=0.12
  Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
  Using cached wcwidth-0.1.9-py2.py3-none-any.whl (19 kB)
Collecting more-itertools>=4.0.0
  Using cached more_itertools-8.2.0-py3-none-any.whl (43 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting six
  Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Building wheels for collected packages: pytest
  Building wheel for pytest (PEP 517) ... done
  Created wheel for pytest: filename=pytest-5.4.1.dev146+g9bf3efb8e-py3-none-any.whl size=249018 sha256=d4d4099b58b94818996ff3ade981162d6f0ada0423be1c21049233eda2928043
  Stored in directory: /tmp/pip-ephem-wheel-cache-o2ts6g20/wheels/71/a1/e6/24d12ec774c84a6b2173782c0ca120faa2de855d11ff1e6c5d
Successfully built pytest
Installing collected packages: attrs, py, pyparsing, six, packaging, pluggy, wcwidth, more-itertools, pytest
Successfully installed attrs-19.3.0 more-itertools-8.2.0 packaging-20.3 pluggy-0.13.1 py-1.8.1 pyparsing-2.4.7 pytest-5.4.1.dev146+g9bf3efb8e six-1.14.0 wcwidth-0.1.9
altendky@p1:~/repos/ciborg$ venv/bin/pip list
Package               Version                 Location                       
--------------------- ----------------------- -------------------------------
appdirs               1.4.3                   
attrs                 19.3.0                  
ciborg                0.1                     /home/altendky/repos/ciborg/src
click                 7.1.1                   
coverage              5.1                     
distlib               0.3.0                   
filelock              3.0.12                  
importlib-resources   1.4.0                   
marshmallow           3.5.1                   
marshmallow-polyfield 5.9                     
more-itertools        8.2.0                   
mypy                  0.770                   
mypy-extensions       0.4.3                   
packaging             20.3                    
pep517                0.8.2                   
pip                   20.0.2                  
pluggy                0.13.1                  
py                    1.8.1                   
pyparsing             2.4.7                   
pyrsistent            0.16.0                  
pytest                5.4.1.dev146+g9bf3efb8e 
pytest-cov            2.8.1                   
PyYAML                5.3.1                   
setuptools            45.2.0                  
six                   1.14.0                  
toml                  0.10.0                  
tox                   3.14.6                  
typed-ast             1.4.1                   
typing-extensions     3.7.4.2                 
virtualenv            20.0.18                 
wcwidth               0.1.9                   
wheel                 0.34.2                  
altendky@p1:~/repos/ciborg$ cat x.py
a = 'a\n'*500
b = 'b\n'*500

n = 52


def test_x():
    assert a[:n] == b[:n]

def test_y():
    assert a[:n + 1] == b[:n + 1]
altendky@p1:~/repos/ciborg$ venv.test/bin/pytest -vv x.py
====================================================================================================================== test session starts =======================================================================================================================
platform linux -- Python 3.8.2, pytest-5.4.1.dev146+g9bf3efb8e, py-1.8.1, pluggy-0.13.1 -- /home/altendky/repos/ciborg/venv.test/bin/python
cachedir: .pytest_cache
rootdir: /home/altendky/repos/ciborg, inifile: pytest.ini
collected 2 items                                                                                                                                                                                                                                                

x.py::test_x FAILED                                                                                                                                                                                                                                        [ 50%]
x.py::test_y FAILED                                                                                                                                                                                                                                        [100%]

============================================================================================================================ FAILURES ============================================================================================================================
_____________________________________________________________________________________________________________________________ test_x _____________________________________________________________________________________________________________________________

    def test_x():
>       assert a[:n] == b[:n]
E       AssertionError: assert 'a\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\n' == 'b\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\n'
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a

x.py:8: AssertionError
_____________________________________________________________________________________________________________________________ test_y _____________________________________________________________________________________________________________________________

    def test_y():
>       assert a[:n + 1] == b[:n + 1]
E       AssertionError: assert ('a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a') == ('b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b\n'\n 'b')
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         - b
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a

x.py:11: AssertionError
==================================================================================================================== short test summary info =====================================================================================================================
FAILED x.py::test_x - AssertionError: assert 'a\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\n' == 'b\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\n'
FAILED x.py::test_y - AssertionError: assert ('a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n 'a\n'\n '...
======================================================================================================================= 2 failed in 0.05s ========================================================================================================================

Activity

  1. altendky commented on Apr 30, 2020

    @altendky
    altendky
    on Apr 30, 2020
    MemberAuthor

    No fix there in the PR but a test anyways in the suite.

  2. altendky commented on May 1, 2020

    @altendky
    altendky
    on May 1, 2020
    MemberAuthor

    I traced this down to a pprint.pformat() call in _pytest._io.saferepr.safeformat(). Passing width=99999 for example improves my case as does just using repr(). I'm looking into existing tests related to this to see what the existing requirements are.

    pytest/src/_pytest/_io/saferepr.py

    Lines 64 to 72 in b7ad4c2

    def safeformat(obj: Any) -> str:
    """return a pretty printed string for the given object.
    Failing __repr__ functions of user instances will be represented
    with a short exception info.
    """
    try:
    return pprint.pformat(obj)
    except Exception as exc:
    return _format_repr_exception(exc, obj) 

    Python 3.8.2 (default, Mar  7 2020, 20:18:24) 
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pprint
>>> s26 = 's\n' * 26
>>> s27 = 's\n' * 27
>>> pprint.pformat(s26)
"'s\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\ns\\n'"
>>> pprint.pformat(s27)
"('s\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n'\n 's\\n')"

  3. altendky commented on May 1, 2020

    @altendky
    altendky
    on May 1, 2020
    MemberAuthor

    pprint.pformat() sorts dicts, so repr() will not match present behavior (including failing a py35 test in CI due to lack of maintaining insertion order). pprint.pformat(obj, width=1000000) won't match because then (of course...) pprint won't wrap stuff (though this does pass CI).

    So, one probably bad idea would be to special case strings. Or perhaps I'm looking at the wrong thing as the error and it's not that pprint is bad here but rather that it's getting corrupted downstream such that it doesn't get presented as pprint intended. Maybe related to the newline rewrites because of other pieces I forget the names of at the moment.

    And of course the situation I really care about is PyCharm's diff link so I'll have to dig into what that actually supports.

  4. Zac-HD
    added
    topic: reportingrelated to terminal output and user-facing messages and errors
    topic: rewriterelated to the assertion rewrite mechanism
    on May 10, 2020

  5. Zac-HD commented on Jul 22, 2026

    @Zac-HD
    Zac-HD
    on Jul 22, 2026
    Member

    Closing as resolved: reproduced the repro on current main and the corrupted pprint-tuple-style summary for long string diffs no longer occurs — assertrepr_compare now builds the summary via saferepr rather than pprint.pformat. Related long-string-diff work landed in #14425 (closing #6757, adds assertion_text_diff_style). Reopen with a fresh repro if a variant persists.

  6. Zac-HD
    closed this as completedon Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic: reportingrelated to terminal output and user-facing messages and errorstopic: rewriterelated to the assertion rewrite mechanism

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions