Skip to content

SingleProxy: Failed to create httpx.Client in Python 3.13 #577

Description

@EgodPrime

Describe the bug
A clear and concise description of what the bug is.

To Reproduce

from scholarly import ProxyGenerator

pg = ProxyGenerator()
http_proxy_url = "http://anystr"
success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)

The above code outputs:

Traceback (most recent call last):
  File "/root/tracefuzz/test_scholarly.py", line 5, in <module>
    success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 124, in SingleProxy
    proxy_works = self._use_proxy(http=http, https=https)
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 215, in _use_proxy
    self._new_session(proxies=proxies)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 491, in _new_session
    self._session = httpx.Client(**init_kwargs)
                    ~~~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: Client.__init__() got an unexpected keyword argument 'proxies'

Expected behavior
success should be True.

Screenshots

Image

In this Python version, httpx.Client receives proxy (URL|str|Proxy) rather than proxies

Desktop (please complete the following information):

  • Proxy service: SingleProxy
  • python version: [3.13.4]
  • OS: [Ubuntu 22.04.5 LTS]
  • Version [1.7.11]

Do you plan on contributing?
No, but I think it's a piece of cake to fix this issue, so the developer can make it.

# scholarly/_proxy_generator.py
# ...
if self._proxy_works:
            init_kwargs["proxies"] = proxies #.get("http", None)
            """==================================
                Add code like this
            =================================="""
            if sys.version_info >= (3, 13):
                init_kwargs["proxy"] = proxies.get("http") 
                init_kwargs.pop("proxies", None) 
             """==================================
             =================================="""
            self._proxies = proxies
            if self.proxy_mode is ProxyMode.SCRAPERAPI:
                # SSL Certificate verification must be disabled for
                # ScraperAPI requests to work.
                # https://www.scraperapi.com/documentation/
                init_kwargs["verify"] = False
        self._session = httpx.Client(**init_kwargs)
        self._webdriver = None

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions