Debugger breaks when patching json.loads

[ksaito1125]

The code below runs correctly in pytest.
When I run the same code from VS Code's decoder, it doesn't return from the patch call.

import json


def sample(json_data):
    return json.loads(json_data)

from unittest.mock import MagicMock, patch


def test_sample():
    data = {'a1': 'b', 'a2': 'c'}
    json_data = json.dumps(data)
    with patch('json.loads', return_value='aa'): # Step execution does not come back
        assert 'aa' == sample(json_data)

def test_sample_mocker(mocker):
    data = {'a1': 'b', 'a2': 'c'}
    json_data = json.dumps(data)
    mocker.patch('json.loads', return_value='aa') # Step execution does not come back
    print('debug=' + sample(json_data))
    assert 'aa' == sample(json_data)

The execution result of pytest is good as follows.

$ pytest test_sample.py 
============================================================ test session starts =============================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /workspaces/sapapi
plugins: cov-3.0.0, mock-3.7.0
collected 2 items                                                                                                                            

test_sample.py ..                                                                                                                      [100%]

============================================================= 2 passed in 0.05s ==============================================================

The execution result of VS Code is as follows.

[benevolentblend]

I'm running into this issue as well. For now, I'm able to work around it by patching an intermediate module method. Not ideal:

temp.py:

import json

def custom_json_load(data):
    return json.loads(data)

Code:

import temp

...

json_data = temp.custom_json_load(data)

Unit Test:

mocker.patch('temp.custom_json_load', return_value='aa')

---

Edit: Corrected the imports from my example

[fabioz]

I'll investigate.

[fabioz]

What happens in this scenario is that the debugger itself uses json.loads, so, when the patching is done, everything that uses json.loads becomes broken, including the debugger...

Unfortunately the debugger (being a debugger written in Python) really needs those bits working (as well as many other things from the standard library), so, right now I'm closing as won't fix (the workaround is creating a different function that calls json.loads and then mock that function instead of mocking the standard library).

An actual fix for this would mean that the debugger couldn't rely on the standard library at all (which would mean that it'd need to be written in a different language or we'd have to copy large portions of the standard library... even just using third party modules wouldn't work as the problem would remain the same where a user can monkey-patch that third party module -- maybe vendoring could be considered, but vendoring for all the platforms that the debugger itself supports isn't straightforward either).

[davidshepherd7]

Hi, we also ran into this issue so just wanted to share some thoughts.

In our case the mock was an autouse mock of sleep which breaks pydevd. We were able to work around the issue by disabling the mock if the DEBUGPY_RUNNING variable is set. This works for us because the mock isn't needed for the tests to work, it only exists to prompt people to consider mocking sleep if their test use a sleep.

An actual fix for this would mean that the debugger couldn't rely on the standard library at all (which would mean that it'd need to be written in a different language or we'd have to copy large portions of the standard library... even just using third party modules wouldn't work as the problem would remain the same where a user can monkey-patch that third party module -- maybe vendoring could be considered, but vendoring for all the platforms that the debugger itself supports isn't straightforward either).

In case it's useful: I think it's possible to bypass the mocking by using an aliasing module for debugpy/pydevd's stdlib imports because patch applies to the place where the object is looked up not the object itself. Something like this seems to work to avoid using a mock in my quick tests:

stdlib_aliases.py

from json import loads as json_loads

In other code:

from stdlib_aliases import time_sleep
from stdlib_aliases import json_loads

# Local imports don't work - they pick up the mock

That's quite a pain if you use stdlib a lot, but maybe worth it?