Is serialisation deterministic? #12067
Replies: 1 comment
|
I would distinguish repeatable in a particular case from canonical serialization. For ordinary models containing only ordered values, Pydantic preserves model-field order and Python dictionaries preserve insertion order, so repeated But Pydantic does not turn arbitrary data into a canonical representation. Output can vary when the value graph contains, for example:
If the output will be hashed, signed, cached by bytes, or used as an idempotency key, normalize unordered values and perform an explicit canonical JSON step: import json
data = model.model_dump(mode="json")
canonical = json.dumps(
data,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")For a set-like field, make its ordering explicit before that step, preferably in a serializer: from pydantic import BaseModel, field_serializer
class Example(BaseModel):
tags: set[str]
@field_serializer("tags")
def serialize_tags(self, tags: set[str]) -> list[str]:
return sorted(tags)Also note that So: deterministic for common ordered inputs in the same configuration, yes; a blanket canonical-output guarantee for every supported value, no. Reference: https://docs.pydantic.dev/latest/concepts/serialization/ If this solves your issue, please consider marking this comment as the answer! |
Uh oh!
There was an error while loading. Please reload this page.
I found this question, which focused on
model_dump_jsonspecifically (and was unanswered anyway).More generally, assuming no custom serialiser is nondeterministic, are the outputs of
model_dumpandmodel_dump_json(and e.g.TypeAdapter.dump_json) deterministic?All reactions