Server returns 415 for Content-Types with parameters (e.g., application/json; charset=utf-8)

[ElanHR]

Summary

The server-side codec lookup in _protocol_connect.codec_name_from_content_type strips only the application/ (or application/connect+) prefix and uses the rest verbatim as the codec name. When a client sends a Content-Type with parameters — e.g. application/json; charset=utf-8 — the lookup key becomes json; charset=utf-8, no codec matches, and _server_async.py:213 (and the _server_sync.py counterparts at :306/:437) raises HTTPException(HTTPStatus.UNSUPPORTED_MEDIA_TYPE).

Per RFC 9110 §8.3.2 the charset parameter is part of the standard media-type grammar and the value is case-insensitive so servers are expected to tolerate it.

Reproduction

# Works:
curl -X POST -H 'Content-Type: application/json' -d '{}' \
  https://my-server/<service>/<method>
# → 401/200 — codec resolves

# Fails:
curl -X POST -H 'Content-Type: application/json; charset=utf-8' -d '{}' \
  https://my-server/<service>/<method>
# → HTTP 415 Unsupported Media Type

Hits in production when a browser or proxy adds ; charset=utf-8 to the Connect-ES request (we observed this on a Connect-ES --> Connect-Python path even though @connectrpc/connect's request-header.js builds the Content-Type without parameters).

Expected behavior

Something matching the reference Go implementation, ie. canonicalizeContentType calls mime.ParseMediaType, discards/canonicalizes parameters, and matches on the bare media type.

Suggested fix

In _protocol_connect.codec_name_from_content_type, parse the input via Python's media-type parser before stripping the prefix:

from email.message import Message

def codec_name_from_content_type(content_type: str, *, stream: bool) -> str:
    msg = Message()
    msg["content-type"] = content_type
    base = msg.get_content_type()  # e.g. "application/json"
    prefix = (
        CONNECT_STREAMING_CONTENT_TYPE_PREFIX
        if stream
        else CONNECT_UNARY_CONTENT_TYPE_PREFIX
    )
    if base.startswith(prefix):
        return base[len(prefix):]
    return base

Thanks for all the work on the project. Happy to send a PR if it's useful. :)

[anuraaga]

Hi @ElanHR - I sent #235 which should verify that the basic charset form like your example should be working already with a default server configuration. Do you see anything different about your usage? Just to confirm.

That being said, this still looks worth fixing to be more general. I had noticed connect-go uses a non-parsing approach on the client side

https://github.com/connectrpc/connect-go/blob/a5a6c30f3776b06ae05a66ab3bdd2d60c46db6db/option.go#L630

but missed that it does do the canonicalization on the server side so will send a fix for that.

[ElanHR]

After further debugging we found the previous claim of "observed in Connect-ES traffic" in my original post isn't correct. Our production 415 turned out to be a separate issue (the server received the request with no content-type header at all. Empty Content-Type also fails the codec lookup and returns 415, which is why the symptom looked identical). Apologies for the mischaracterization! 😅

That said, he underlying connect-python issue still reproduces on its own:

curl -X POST -H 'Content-Type: application/json; charset=utf-8' -d '{}' \
  https://my-server/<service>/<method>
# → HTTP 415

So RFC 9110 §8.3.2 still treats charset as part of the standard media-type grammar, and connect-go handles this via canonicalizeContentType -->mime.ParseMediaType, so the gap with the reference implementation stands but my framing was definitely incorrect.

[anuraaga]

Thanks @ElanHR - it may not be worth debugging too much since it makes sense to do looser handling (support multiple spaces for example), but I'm still wondering what part of that misses our currently too strict handling, the header looks the same as what I included in the unit test. By the way are you checking with the latest version?

Anyways no worries if we can't get to the bottom of that.

[ElanHR]

Ah I was a bit sloppy when rewriting the command used for testing to "be a minimal approach" and dropped the capitalization (our client sends "UTF-8" instead of "utf-8") which does not exactly match against

connect-python/src/connectrpc/_codec.py

Line 13 in 4859bac

CODEC_NAME_JSON_CHARSET_UTF8 = "json; charset=utf-8"

The following is what I should have sent - tested against an unmodified v0.9.0 server (just noticed v0.10.0 released!):

curl -X POST -H 'Content-Type: application/json; charset=UTF-8' -d '{}' \
  https://my-server/<service>/<method>

That all said , #235 looks like it should resolve all of these issues. :)

[anuraaga]

Thanks for following up @ElanHR - so if I understand correctly, you had an issue with missing content type header, and the improvement needed for matching i.e. uppercase was a red herring when debugging that? If that's right and it means you are not currently affected by it, I don't think we will rush out a patch with the fix in #235 since I think it has quite limited impact. But let me know if you do need it.

[ElanHR]

Not quite - I am affected by this but absolutely please don't feel any need to rush a patch - happy to wait for the next release on this! :)

Something in the network/istio mesh is normalizing headers to 'Content-Type: application/json; charset=UTF-8' which while technically up to spec (should be insensitive to case/spacing) causes the CODEC_NAME_JSON_CHARSET_UTF8 mismatch described.

In the meantime I was able to add a piece of middleware to strip the Content Type params to the ConnectRPC endpoint so I am in no way blocked by this.