email.message.Message (RFC 2822 parser) used for HTTP Content-Type detection causes FastAPI to parse application/x-www-form-urlencoded+json bodies as JSON, creating inconsistent behavior with application middleware
#15201
Unanswered
subhashdasyam
asked this question in
Questions
Replies: 1 comment
-
|
Just notes for me future: I think this should be investigated and probably addressed at some point in the future ( |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
First Check
Commit to Help
Example Code
email.message.Messageis an RFC 2822 email/MIME header parser, not an HTTP/RFC 7231 parser. It fully implements the MIME Structured Syntax Suffix convention (RFC 6839), which says any type with a+jsonsuffix contains JSON-encoded data. As a result, FastAPI treats all of the following as JSON bodies:application/x-www-form-urlencoded+jsonapplication/octet-stream+jsonapplication/vnd.anything+jsonNone of these are registered media types.
application/x-www-form-urlencoded+jsonin particular is semantically contradictory — URL-encoded form data cannot simultaneously be JSON-encoded.What I expected to happen
application/x-www-form-urlencoded+jsonshould not triggerawait request.json(). It is not a real content type. FastAPI should either treat it as raw bytes or reject it with HTTP 415 Unsupported Media Type, just as it does for plainapplication/x-www-form-urlencodedon a JSON-body endpoint.What is currently happening
Any string matching
application/*+json— no matter how semantically invalid the base subtype is — causes FastAPI to callrequest.json()and deliver a parsed Python dict to the endpoint. The endpoint has no way to know the Content-Type was unusual.Why this creates real problems at the application level
The core issue is inconsistency between FastAPI's body parsing decision and what the rest of the application stack assumes.
Application middleware commonly branches on Content-Type to implement different behavior for JSON API calls versus form submissions. This is a standard pattern when an API serves both programmatic clients and browser forms:
In every pattern above,
application/x-www-form-urlencoded+jsongoes down the non-JSON branch in the middleware (becausect.startswith("application/json")is False), while FastAPI's body parser takes the JSON branch (because the email parser sees+json).The result: the endpoint receives a fully parsed JSON dict, but the middleware applied the wrong set of rules to get it there.
Concrete impact:
1. Auth requirement bypass (shown in example code above):
Middleware enforces strong auth for
application/jsonrequests. Usingapplication/x-www-form-urlencoded+json, a client with only a session cookie can call a JSON API endpoint that requires an admin token. The session auth path has no role check — it was never designed to handle JSON API payloads, only browser form submissions.2. Input validation middleware bypass:
If application middleware validates the structure or contents of
application/jsonrequest bodies (e.g., checking for disallowed fields, enforcing field-level business rules before the request reaches the endpoint), those checks are skipped entirely when the same JSON payload arrives with a+json-suffixed content type.3. Unexpected behavior with per-route strict_content_type:
An endpoint decorated with
strict_content_type=Truecorrectly rejects requests with no Content-Type. But a request withContent-Type: application/octet-stream+jsonpasses through —strict_content_typeonly guards the no-Content-Type path, not the+jsonsuffix path.4. Silent parsing of semantically invalid content types:
A client that accidentally sends
Content-Type: application/x-www-form-urlencodedwith a JSON body gets a 422. A client that accidentally sendsContent-Type: application/x-www-form-urlencoded+jsonwith a JSON body gets a 200. This inconsistency makes debugging hard: the same malformed content type produces different outcomes depending on whether the+jsonsuffix is present.Content-Type classification inconsistency table
ct.startswith("application/json")application/jsonapplication/x-www-form-urlencodedapplication/x-www-form-urlencoded+jsonapplication/octet-stream+jsonapplication/vnd.api+jsontext/plain+jsonProposed fix
Replace the
email.message.Messageblock with direct string parsing and explicitly reject known non-JSON base subtypes:What this changes:
application/json— still JSON (exact match)application/vnd.api+json,application/ld+json,application/problem+json— still JSON (legitimate structured-syntax types)application/x-www-form-urlencoded+json— now bytes (was incorrectly JSON)application/octet-stream+json— now bytes (was incorrectly JSON)import email.messagefrom the HTTP request handling path entirelyOperating System
Linux
Operating System Details
Linux 6.17.9
FastAPI Version
0.135.1
Pydantic Version
2.12.5
Python Version
3.12.3
Additional Context
Why
email.message.Messageis the wrong tool hereemail.message.Message(Python'semailstdlib) is designed to parse RFC 2822 message headers — email headers. HTTPContent-Typeheaders follow RFC 7231, which has different quoting rules, parameter handling, and case-sensitivity semantics. Using the email parser for HTTP headers works for common cases (application/json,application/json; charset=utf-8) but diverges on edge cases like structured syntax suffixes, non-standard parameter quoting, and folded header values.The simplest correct replacement is direct string manipulation: split at
;, strip, lowercase, check prefix and suffix. No external parser needed.Note on
strict_content_typeThe existing
strict_content_type=Trueroute parameter does not mitigate this issue. It only guards the case where no Content-Type header is present. Once a Content-Type value exists — evenapplication/x-www-form-urlencoded+json—strict_content_typehas no effect on the+jsonsuffix detection path.Beta Was this translation helpful? Give feedback.
All reactions