Requests works, but httpx not - Help Needed

[klacol]

Hello,

i am working on an interface to a smart meter device for an integration in Home Assistant. The device has an API that can be requested via https.

I was working with the python lib requests and that works. But the integration uses the lib httpx. But with this lib I cannot get the connection. To the same device, within the same network, same user, password, certificate.

Where can be the difference?

I wrote a script, that tries to make the connection with both libs, but only the one made with request works.

I have to say, that between me and the device, there is small NAT Router; but could this affect the httpx-connection and it works with requests?

This is the script
https://gist.github.com/klacol/d8b30cf0b08267c2366bed86b42302ee

This is the log from the first connection with requests; works like a charm:

INFO:main:Theben Connexa HTTP Client
INFO:main:This is a connection test for the Theben Connexa Smart-Meter Gateway
DEBUG:main:Debug logging is enabled
INFO:main:Connecting to 10.11.120.2:443...
INFO:main:I have downloaded the public certificate from the IP address 10.11.120.2 and analyzed it.
INFO:main:The certificate is saved in 10.11.120.2_public_cert.pem.
INFO:main:The certificate is valid for this name (SAN): ethe0300186023.sm
INFO:main:The certificate is only validated successfully if the call can be made using this name.
INFO:main:Therefore, please access the device directly via ethe0300186023.sm or set up an alias in the DNS server: ethe0300186023.sm=10.11.120.2
INFO:main:Calling: POST https://ethe0300186023.sm/smgw/m2m/
INFO:main:Using username: 30320426231LGZ0067558611
INFO:main:Using body: {'method': 'smgw-info'}
Connect with the REQUESTS library
REQUESTS Version: 2.32.3
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): ethe0300186023.sm:443
DEBUG:urllib3.connectionpool:https://ethe0300186023.sm:443 "POST /smgw/m2m/ HTTP/11" 401 16
DEBUG:urllib3.connectionpool:https://ethe0300186023.sm:443 "POST /smgw/m2m/ HTTP/11" 307 8
DEBUG:urllib3.connectionpool:https://ethe0300186023.sm:443 "POST /smgw/m2m/30320426231lgz0067558611.sm/json HTTP/11" 401 0
DEBUG:urllib3.connectionpool:https://ethe0300186023.sm:443 "POST /smgw/m2m/30320426231lgz0067558611.sm/json HTTP/11" 200 9905
Status Code: 200
Firmware Version: v3.80.0-cc

This is the log from the second connection with https; has errors:

Connect with the HTTPX library
HTTPX Version: 0.28.1
DEBUG:httpcore.connection:connect_tcp.started host='ethe0300186023.sm' port=443 local_address='0.0.0.0' timeout=30.0 socket_options=[(65535, 8, 2)]
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.sync.SyncStream object at 0x000001BE4C334F10>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0x000001BE4C30CD40> server_hostname='ethe0300186023.sm' timeout=30.0
DEBUG:httpcore.connection:start_tls.failed exception=ConnectError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)'))
DEBUG:httpcore.connection:retry.started ssl_context=<ssl.SSLContext object at 0x000001BE4C30CD40> server_hostname='ethe0300186023.sm' timeout=30.0
DEBUG:httpcore.connection:retry.complete
DEBUG:httpcore.connection:connect_tcp.started host='ethe0300186023.sm' port=443 local_address='0.0.0.0' timeout=30.0 socket_options=[(65535, 8, 2)]
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.sync.SyncStream object at 0x000001BE4C335A50>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0x000001BE4C30CD40> server_hostname='ethe0300186023.sm' timeout=30.0
DEBUG:httpcore.connection:start_tls.failed exception=ConnectError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)'))
DEBUG:httpcore.connection:retry.started ssl_context=<ssl.SSLContext object at 0x000001BE4C30CD40> server_hostname='ethe0300186023.sm' timeout=30.0
DEBUG:httpcore.connection:retry.complete
DEBUG:httpcore.connection:connect_tcp.started host='ethe0300186023.sm' port=443 local_address='0.0.0.0' timeout=30.0 socket_options=[(65535, 8, 2)]
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.sync.SyncStream object at 0x000001BE4C335E90>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0x000001BE4C30CD40> server_hostname='ethe0300186023.sm' timeout=30.0
DEBUG:httpcore.connection:start_tls.failed exception=ConnectError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)'))
Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)

I know, it is complicated to help, without being able to replay this.

Any help appreciated

Thanks!

[widechaos]

Your log shows local_address="0.0.0.0", custom socket_options, and connection retries. Those settings are absent from the current linked gist, so I would check whether the version producing the log passes an explicit HTTPTransport to the client.

In HTTPX 0.28.1, an explicit transport does not inherit Client(verify=ssl_context). Client._init_transport returns the supplied transport unchanged; HTTPTransport creates its own SSL context. If your running code supplies a transport, put the certificate configuration there:

ssl_context = ssl.create_default_context(cafile=cert_file)
transport = httpx.HTTPTransport(
    verify=ssl_context,
    local_address="0.0.0.0",
    retries=2,
    # Keep any socket_options required by your existing setup here.
)
with httpx.Client(
    transport=transport,
    auth=httpx.DigestAuth(username, password),
    headers=headers,
    timeout=30.0,
    follow_redirects=True,
) as client:
    response = client.post(url, json=body)

If you do not need a custom transport, the current gist's Client(verify=ssl_context, ...) is already the correct placement. Keep the SAN hostname in the URL and use a certificate/CA verified as belonging to your device.

I checked the context wiring locally on HTTPX 0.28.1: a client-created transport used the supplied context; an explicit transport without verify did not; adding verify to that transport restored it. This was a configuration check, not a TLS handshake test against your meter. If the current gist still produces that error without a custom transport, please share the exact client/transport setup used for that run, with credentials removed. HTTPX SSL configuration.