requests 2.32.3 with IPv6 link local address fails with error: [Errno -2] Name or service not known #6735

pytech66 · 2024-06-06T18:29:52Z

When using an IPv6 link local address with requests > v2.31.0, seeing an error [Errno -2] Name or service not known. The same works with requests 2.31.0.
urllib3 2.2.1 also works successfully with IPv6 link local address .

urllib3 2.2.1 with IPv6 link local address

>>> import urllib3
>>> http = urllib3.PoolManager(cert_reqs='CERT_NONE')
>>> res = http.request("GET", "https://[fe80::5eed:8cff:fe00:0da4%ens192]/redfish/v1")
/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connectionpool.py:1103: InsecureRequestWarning: Unverified HTTPS request is being made to host 'fe80::5eed:8cff:fe00:0da4%ens192'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
>>> res.status
200

requests 2.31.0 with IPv6 link local address

>>> import requests
>>> res = requests.get("https://[fe80::5eed:8cff:fe00:0da4%ens192]/redfish/v1", verify=False)
>>> res.status_code
200

Expected Result

Successful connection with versions 2.32.x

Actual Result

>>> import requests
>>> res = requests.get("https://[fe80::5eed:8cff:fe00:0da4%ens192]/redfish/v1", verify=False)
Traceback (most recent call last):
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connection.py", line 205, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x716b63d403d0>: Failed to resolve 'fe80::5eed:8cf00:0da4%25ens192' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/new-requests/venv/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/root/new-requests/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='fe80::5eed:8cff:fe00:0da4%25ens192', port=443): Max retries exceeded with urredfish/v1 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x716b63d403d0>: Failed to resolve 'fe80::5eed::fe00:0da4%25ens192' ([Errno -2] Name or service not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/new-requests/venv/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/root/new-requests/venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/root/new-requests/venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/root/new-requests/venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/root/new-requests/venv/lib/python3.10/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='fe80::5eed:8cff:fe00:0da4%25ens192', port=443): Max retries exceeded with: /redfish/v1 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x716b63d403d0>: Failed to resolve 'fe80::5ecff:fe00:0da4%25ens192' ([Errno -2] Name or service not known)"))

Reproduction Steps

import requests
res = requests.get("https://[fe80::5eed:8cff:fe00:0da4%ens192]/redfish/v1", verify=False)

System Information

$ python -m requests.help

{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.12"
  },
  "platform": {
    "release": "6.5.0-27-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.32.3"
  },
  "system_ssl": {
    "version": "30000020"
  },
  "urllib3": {
    "version": "2.2.1"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

The text was updated successfully, but these errors were encountered:

mattiaswal · 2024-06-27T08:52:18Z

I worked around the bug with:

# Workaround for bug in requests 2.32.x: https://github.com/psf/requests/issues/6735
def requests_workaround(method, url, json, headers, auth, verify=False):
    # Create a session
        session=requests.Session()

        # Prepare the request
        request=requests.Request(method, url, json=json, headers=headers, auth=auth)
        prepared_request=session.prepare_request(request)
        prepared_request.url=prepared_request.url.replace('%25', '%')
        return session.send(prepared_request, verify=verify)

def requests_workaround_put(url, json, headers, auth, verify=False):
    return requests_workaround('PUT', url, json, headers, auth, verify=False)

def requests_workaround_delete(url, headers, auth, verify=False):
    return requests_workaround('DELETE', url, None, headers, auth, verify=False)

def requests_workaround_post(url, json, headers, auth, verify=False):
    return requests_workaround('POST', url, json, headers, auth, verify=False)

def requests_workaround_get(url, headers, auth, verify=False):
    return requests_workaround('GET', url, None, headers, auth, verify=False)

xiachen-rh · 2024-12-24T06:48:51Z

Hello, I hit the problem too when using python3-requests-2.32.3-2.
Reproduced steps (# python testre.py )

import requests
res = requests.get("http://[fe80::a9fe:a9fe%25enp3s0]/openstack", verify=False)

I found the problem is caused by the changes in this commit c0813a2

requests/adapters.py
def _urllib3_request_context(
...
    parsed_request_url = urlparse(request.url)
...
    host_params = {
        "scheme": scheme,
        "host": parsed_request_url.hostname,
        "port": port,
    }

When the url contains IPv6 address, urllib.urlparse.hostname does not contain the chars "[" "]", but urllib3.util.url._normalize_host matches ipv6 host with chars "[" "]". so the exception is thrown out.

The fix could be using urllib3.util.parse_url() instead of urlparse()

parsed_request_url = parse_url(request.url) # from urllib3.util

OR using netloc instead of hostname

    host_params = { 
        "scheme": scheme,
        "host": parsed_request_url.netloc,
        "port": port,
    }

@sigmavirus24 @nateprewitt What do you think of it?

Here are my system information

# python -m requests.help
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.12.8"
  },
  "platform": {
    "release": "6.12.0-35.el*.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.32.3"
  },
  "system_ssl": {
    "version": "30200020"
  },
  "urllib3": {
    "version": "1.26.19"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

and my debug logs, I adding some print messages into code to debug the issue.

# python testre.py 
url._normalize_host host: [fe80::a9fe:a9fe%25enp3s0]
url._normalize_host is_ipv6: <re.Match object; span=(0, 26), match='[fe80::a9fe:a9fe%25enp3s0]'>
checking request http://[fe80::a9fe:a9fe%25enp3s0]/openstack
checking send
checking adapter: <requests.adapters.HTTPAdapter object at 0x7f4ddf5d0ad0>
if using urllib.urlparse: ParseResult(scheme='http', netloc='[fe80::a9fe:a9fe%25enp3s0]', path='/openstack', params='', query='', fragment='')
requests.adapter parsed_request_url: ParseResult(scheme='http', netloc='[fe80::a9fe:a9fe%25enp3s0]', path='/openstack', params='', query='', fragment='')
**the problem is here: {'scheme': 'http', 'host': 'fe80::a9fe:a9fe%25enp3s0', 'port': None}**
HttpConnectionPool init: fe80::a9fe:a9fe%25enp3s0
###init host: fe80::a9fe:a9fe%25enp3s0
url._normalize_host host: fe80::a9fe:a9fe%25enp3s0
url._normalize_host is_ipv6: None
connection host: fe80::a9fe:a9fe%25enp3s0
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/socket.py", line 976, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 716, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 417, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib64/python3.12/http/client.py", line 1336, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.12/http/client.py", line 1382, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.12/http/client.py", line 1331, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.12/http/client.py", line 1091, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.12/http/client.py", line 1035, in send
    self.connect()
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f4ddf5d0980>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/requests/adapters.py", line 672, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 802, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/util/retry.py", line 594, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='fe80::a9fe:a9fe%25enp3s0', port=80): Max retries exceeded with url: /openstack (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4ddf5d0980>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/testre.py", line 2, in <module>
    res = requests.get("http://[fe80::a9fe:a9fe%25enp3s0]/openstack", verify=False)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 590, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 705, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/adapters.py", line 705, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='fe80::a9fe:a9fe%25enp3s0', port=80): Max retries exceeded with url: /openstack (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4ddf5d0980>: Failed to establish a new connection: [Errno -2] Name or service not known'))

When using urllib3.util.parse_url, the logs are

# python testre.py 
url._normalize_host host: [fe80::a9fe:a9fe%25enp3s0]
url._normalize_host is_ipv6: <re.Match object; span=(0, 26), match='[fe80::a9fe:a9fe%25enp3s0]'>
checking request http://[fe80::a9fe:a9fe%25enp3s0]/openstack
checking send
checking adapter: <requests.adapters.HTTPAdapter object at 0x7f2635e258e0>
url._normalize_host host: [fe80::a9fe:a9fe%25enp3s0]
url._normalize_host is_ipv6: <re.Match object; span=(0, 26), match='[fe80::a9fe:a9fe%25enp3s0]'>
if using urllib3.util.parse_url: http://[fe80::a9fe:a9fe%enp3s0]/openstack
requests.adapter parsed_request_url: http://[fe80::a9fe:a9fe%enp3s0]/openstack
the problem is fixed: {'scheme': 'http', 'host': '[fe80::a9fe:a9fe%enp3s0]', 'port': None}
HttpConnectionPool init: [fe80::a9fe:a9fe%enp3s0]
###init host: [fe80::a9fe:a9fe%enp3s0]
url._normalize_host host: [fe80::a9fe:a9fe%enp3s0]
url._normalize_host is_ipv6: <re.Match object; span=(0, 24), match='[fe80::a9fe:a9fe%enp3s0]'>
connectionpool._normalize_host: fe80::a9fe:a9fe%enp3s0
connection host: fe80::a9fe:a9fe%enp3s0

When using netloc instead of hostname, the logs are

# python testre.py 
url._normalize_host host: [fe80::a9fe:a9fe%25enp3s0]
url._normalize_host is_ipv6: <re.Match object; span=(0, 26), match='[fe80::a9fe:a9fe%25enp3s0]'>
checking request http://[fe80::a9fe:a9fe%25enp3s0]/openstack
checking send
checking adapter: <requests.adapters.HTTPAdapter object at 0x7f0aacced580>
if using urllib.urlparse: ParseResult(scheme='http', netloc='[fe80::a9fe:a9fe%25enp3s0]', path='/openstack', params='', query='', fragment='')
requests.adapter parsed_request_url: ParseResult(scheme='http', netloc='[fe80::a9fe:a9fe%25enp3s0]', path='/openstack', params='', query='', fragment='')
the problem is fixed: {'scheme': 'http', 'host': '[fe80::a9fe:a9fe%25enp3s0]', 'port': None}
HttpConnectionPool init: [fe80::a9fe:a9fe%25enp3s0]
###init host: [fe80::a9fe:a9fe%25enp3s0]
url._normalize_host host: [fe80::a9fe:a9fe%25enp3s0]
url._normalize_host is_ipv6: <re.Match object; span=(0, 26), match='[fe80::a9fe:a9fe%25enp3s0]'>
connectionpool._normalize_host: fe80::a9fe:a9fe%enp3s0
connection host: fe80::a9fe:a9fe%enp3s0

ani-sinha · 2025-01-13T09:42:22Z

Please prioritize this fix.

frenzymadness · 2025-01-17T08:42:50Z

Although the problem is also fixable here, the real one might be in the CPython standard library.

IIUC, the problem is that urllib.parse.urlparse from CPython, when taking a valid link-local IPv6 address (like fe80::a9fe:a9fe%25enp3s0) from a valid URI (like http://[fe80::a9fe:a9fe%25enp3s0]:8888) does not decode the escaped %25 character back to %.

Is that correct?

xiachen-rh · 2025-01-18T01:40:15Z

@frenzymadness the problem is, when the url contains IPv6 address, urllib.urlparse.hostname does not contain the chars "[" "]", but urllib3.util.url._normalize_host matches ipv6 host with chars "[" "]", the exception is thrown out. So the solution could be using netloc instead of hostname.

frenzymadness · 2025-01-18T08:11:34Z

Ok, so the behavior of urllib.urlparse.hostname is correct because IPv6 address is valid without the square brackets outside of URL addresses.

The netloc contains square brackets because it might also contain optional port so the brackets are there to distinguish between these two parts.

You are right that the regular expression in urllib3 requires the IPv6 address to be surrounded with square brackets:

https://github.com/urllib3/urllib3/blob/d86e15d05a5c1b1d6152cecda98b894d3578cbde/src/urllib3/util/url.py#L54

but it does not handle port numbers so something has to split a port from the netloc before _normalize_host is invoked.

And it also seems that the _normalize_host function can handle both decoded/encoded scope identifiers.

xiachen-rh · 2025-03-28T07:31:34Z

@frenzymadness Is there a plan to fix it?

frenzymadness · 2025-03-28T07:36:05Z

@frenzymadness Is there a plan to fix it?

Thanks for asking, but I'm not a maintainer of requests.

ani-sinha · 2025-03-31T15:23:57Z

I worked around the bug with:

# Workaround for bug in requests 2.32.x: https://github.com/psf/requests/issues/6735
def requests_workaround(method, url, json, headers, auth, verify=False):
    # Create a session
        session=requests.Session()

        # Prepare the request
        request=requests.Request(method, url, json=json, headers=headers, auth=auth)
        prepared_request=session.prepare_request(request)
        prepared_request.url=prepared_request.url.replace('%25', '%')
        return session.send(prepared_request, verify=verify)

def requests_workaround_put(url, json, headers, auth, verify=False):
    return requests_workaround('PUT', url, json, headers, auth, verify=False)

def requests_workaround_delete(url, headers, auth, verify=False):
    return requests_workaround('DELETE', url, None, headers, auth, verify=False)

def requests_workaround_post(url, json, headers, auth, verify=False):
    return requests_workaround('POST', url, json, headers, auth, verify=False)

def requests_workaround_get(url, headers, auth, verify=False):
    return requests_workaround('GET', url, None, headers, auth, verify=False)

I tried something in cloud-init along the lines:

diff --git a/cloudinit/url_helper.py b/cloudinit/url_helper.py
index 5adf1f18b..e39dc019a 100644
--- a/cloudinit/url_helper.py
+++ b/cloudinit/url_helper.py
@@ -566,7 +566,21 @@ def readurl(
                     filtered_req_args,
                 )
 
-            response = session.request(**req_args)
+            # Workaround for bug in requests 2.32.x:
+            # See https://github.com/psf/requests/issues/6735
+            # See https://requests.readthedocs.io/en/latest/user/advanced/#prepared-requests
+            req = requests.Request(**req_args)
+            prep = session.prepare_request(req)
+            prep.url = prep.url.replace("%25", "%")
+            settings = session.merge_environment_settings(
+                prep.url, {}, stream, req_args["verify"], req_args["cert"]
+            )
+            send_kwargs = {
+                "timeout": req_args["timeout"],
+                "allow_redirects": allow_redirects,
+            }
+            send_kwargs.update(settings)
+            response = session.send(prep, **send_kwargs)
 
             if check_status:
                 response.raise_for_status()

It does not work. Please feel free to propose something to cloud-init upstream.

urllib.parse.urlparse() does not handle link local ipv6 addresses with port numbers. Use parse_url() from urllib3 instead. Fixes: c0813a2 ("Use TLS settings in selecting connection pool") Fixes: psf#6735 Signed-off-by: Ani Sinha <[email protected]>

urllib.parse.urlparse() does not handle link local ipv6 addresses with port numbers. Use parse_url() from urllib3 instead. Fixes: c0813a2 ("Use TLS settings in selecting connection pool") Fixes: psf#6735 Co-authored-by: Amy Chen <[email protected]> Signed-off-by: Ani Sinha <[email protected]>

ani-sinha · 2025-04-10T11:17:38Z

I worked around the bug with:

snip ...

I tried something in cloud-init along the lines:

diff --git a/cloudinit/url_helper.py b/cloudinit/url_helper.py
index 5adf1f18b..e39dc019a 100644
--- a/cloudinit/url_helper.py
+++ b/cloudinit/url_helper.py
@@ -566,7 +566,21 @@ def readurl(
                     filtered_req_args,
                 )
 
-            response = session.request(**req_args)
+            # Workaround for bug in requests 2.32.x:
+            # See https://github.com/psf/requests/issues/6735
+            # See https://requests.readthedocs.io/en/latest/user/advanced/#prepared-requests
+            req = requests.Request(**req_args)
+            prep = session.prepare_request(req)
+            prep.url = prep.url.replace("%25", "%")
+            settings = session.merge_environment_settings(
+                prep.url, {}, stream, req_args["verify"], req_args["cert"]
+            )
+            send_kwargs = {
+                "timeout": req_args["timeout"],
+                "allow_redirects": allow_redirects,
+            }
+            send_kwargs.update(settings)
+            response = session.send(prep, **send_kwargs)
 
             if check_status:
                 response.raise_for_status()

It does not work. Please feel free to propose something to cloud-init upstream.

I forgot to summarize the issue.
The above change will not fix the issue because the issue is not that the urllib.parse.urlparse() "does not decode the escaped %25 character back to %." as noted above.
The issue is that the CPython library urllib does not not parse the url string to generate a host string that is enclosed with "[" character. Therefore, when we use urllib3 to manage ConnectionPool objects, the urllib3.util.url._normalize_host() which is called from the constructor of ConnectionPool() objects does not detect the host address as IPV6 address and this is where things go wrong.
Clearly, it would seem urllib's urlparse() api is not compatible with urllib3. So since we use urllib3 to manage connection objects, one obvious choice is to use urllib3's parse_url() method instead. It would make things uniform. It would not remove the "[" characters from the host part and seems this api is spec compatible as well.
This is what I have proposed in the above PR I sent out. As a consumer of this library, we have done our part to analyze and suggest a possible solution. Rest, we leave to the maintainer of this library.

pytech66 changed the title ~~requests 2.32.3 with IPv6 link local address fails with socket.gaierror: [Errno -2] Name or service not known~~ requests 2.32.3 with IPv6 link local address fails [Errno -2] Name or service not known Jun 6, 2024

pytech66 changed the title ~~requests 2.32.3 with IPv6 link local address fails [Errno -2] Name or service not known~~ requests 2.32.3 with IPv6 link local address fails with error: [Errno -2] Name or service not known Jun 6, 2024

xiachen-rh mentioned this issue Dec 24, 2024

fix (openstack): Append interface / scope_id for IPv6 link-local metadata address canonical/cloud-init#5419

Merged

5 tasks

ani-sinha linked a pull request Apr 1, 2025 that will close this issue

Use parse_url() from urllib3.util instead of urlparse #6927

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

requests 2.32.3 with IPv6 link local address fails with error: [Errno -2] Name or service not known #6735

requests 2.32.3 with IPv6 link local address fails with error: [Errno -2] Name or service not known #6735

pytech66 commented Jun 6, 2024

mattiaswal commented Jun 27, 2024

xiachen-rh commented Dec 24, 2024 •

edited

Loading

ani-sinha commented Jan 13, 2025

frenzymadness commented Jan 17, 2025

xiachen-rh commented Jan 18, 2025 •

edited

Loading

frenzymadness commented Jan 18, 2025

xiachen-rh commented Mar 28, 2025

frenzymadness commented Mar 28, 2025

ani-sinha commented Mar 31, 2025 •

edited

Loading

ani-sinha commented Apr 10, 2025

requests 2.32.3 with IPv6 link local address fails with error: [Errno -2] Name or service not known #6735

requests 2.32.3 with IPv6 link local address fails with error: [Errno -2] Name or service not known #6735

Comments

pytech66 commented Jun 6, 2024

urllib3 2.2.1 with IPv6 link local address

requests 2.31.0 with IPv6 link local address

Expected Result

Actual Result

Reproduction Steps

System Information

mattiaswal commented Jun 27, 2024

xiachen-rh commented Dec 24, 2024 • edited Loading

ani-sinha commented Jan 13, 2025

frenzymadness commented Jan 17, 2025

xiachen-rh commented Jan 18, 2025 • edited Loading

frenzymadness commented Jan 18, 2025

xiachen-rh commented Mar 28, 2025

frenzymadness commented Mar 28, 2025

ani-sinha commented Mar 31, 2025 • edited Loading

ani-sinha commented Apr 10, 2025

xiachen-rh commented Dec 24, 2024 •

edited

Loading

xiachen-rh commented Jan 18, 2025 •

edited

Loading

ani-sinha commented Mar 31, 2025 •

edited

Loading