Issue: JSON Serialization Behavior Change in 0.28.1 causing HTTP Signature Digest Mismatch

[RHawkins-Fisher]

Summary

After upgrading httpx from 0.25.0 to 0.28.1, HTTP Signatures that rely on a Digest of the request body began failing with a 401 Unauthorized (Authentication Failed) error when interacting with the CyberSource API.

Root Cause Analysis

The issue appears to correspond to a change in how httpx generates the JSON body bytes when using the json argument in client.request().

• Scenario: We generate an HTTP Signature Digest by creating a hash of json.dumps(payload) (using standard Python json module default separators).
• Behavior in 0.25.0: The hash generated from json.dumps(payload) matched the hash of the body sent by httpx.
• Behavior in 0.28.1: The hash generated from json.dumps(payload) did not match the body sent by httpx.

Workaround / Proof

We confirmed the issue by manually serializing the payload to a string using json.dumps(payload) and passing this string to httpx.request(..., content=payload_str).

• When using content=payload_str, the request succeeded with httpx 0.28.1.
• When using json=payload, the request failed corresponding to a digest mismatch.

This indicates that httpx 0.28.1 is producing a JSON byte stream that differs from the standard implementation of json.dumps(payload) (e.g., changes in whitespace/separators or key ordering), whereas 0.25.0 produced a matching stream.

Environment

• Previous Version: httpx==0.25.0 (Working)
• Current Version: httpx==0.28.1 (Failing Digest)
• Python Version: 3.13 (and 3.11)
• OS: Windows

Recommendation / Request

If httpx has intentionally changed default JSON serialization (e.g. to be more compact), it would be helpful to document this behavior, as it breaks compatibility with systems that compute signatures based on standard json.dumps() output. Alternatively, a flag to control serialization behavior would be beneficial.

[amasen02]

Your analysis is spot-on. In httpx, the json= parameter serializes data using compact JSON formatting:

json.dumps(payload, separators=(",", ":"))

The Difference in Wire Bytes

Python's standard library json.dumps() defaults to separators=(", ", ": ") (which includes spaces after commas and colons):

import json
import httpx

payload = {"hello": "world", "count": 123}

# Python default json.dumps:
# b'{"hello": "world", "count": 123}'  <-- Contains whitespace

# httpx wire bytes (compact):
req = httpx.Request("POST", "https://example.com", json=payload)
# b'{"hello":"world","count":123}'      <-- No whitespace

Because cryptographic signatures (like RFC 3230 / RFC 9530 Digest headers used by CyberSource and payment gateways) hash the exact wire bytes, computing the digest against json.dumps(payload).encode() produces a hash of the whitespace-padded string, which mismatches the compact wire bytes transmitted by httpx.

---

Recommended Solutions

1. Idiomatic: Compute the Digest directly from request.content (e.g. via httpx.Auth)

If you compute the signature in an authentication flow or transport, derive the Digest from request.content. By the time the request is handed to an httpx.Auth flow or event hook, request.content has already been materialized into the exact bytes that will cross the network:

import hashlib
import base64
import httpx

class CyberSourceAuth(httpx.Auth):
    def auth_flow(self, request):
        if request.content:
            # Hash the EXACT wire bytes httpx will send
            digest_hash = hashlib.sha256(request.content).digest()
            digest_header = f"SHA-256={base64.b64encode(digest_hash).decode('ascii')}"
            request.headers["Digest"] = digest_header

        # Sign the headers (including Digest, Host, Date, etc.)
        # ... generate Signature header ...
        yield request

client = httpx.Client(auth=CyberSourceAuth())
# Now json=payload works seamlessly!
client.post("https://api.cybersource.com/...", json=payload)

2. Explicit: Pre-serialize with separators=(',', ':') and pass content=

If you need to calculate the signature before calling the client:

import json
import hashlib
import base64
import httpx

body_bytes = json.dumps(payload, separators=(",", ":")).encode("utf-8")
digest_b64 = base64.b64encode(hashlib.sha256(body_bytes).digest()).decode("ascii")

headers = {
    "Content-Type": "application/json",
    "Digest": f"SHA-256={digest_b64}",
    # ... your HTTP signature headers ...
}

response = client.post("https://api.cybersource.com/...", content=body_bytes, headers=headers)

Passing content=body_bytes ensures byte-for-byte predictability and guarantees that the hashed bytes and the transmitted bytes are strictly identical.