nested pydantic cloudevent serialization doesn't work

[sasha-tkachev]

CloudEvent(..., data=CloudEvent(...)).json()

will not work

I will fix it myself

[xSAVIKx]

that's kinda the "default" behavior, but sure, let's wait for a PR 👍

[hozn]

I'm having this issue as well, but in pydantic v2. It works fine in vanilla pydantic, but does not work with the CloudEvent -- or subclasses, as I am using.

In my specific case, I have a Generic subclass for our models.

import uuid
from typing import Generic, TypeVar

from cloudevents.pydantic.v2 import CloudEvent
from pydantic import BaseModel

T = TypeVar("T", bound=BaseModel)


class MyEntity(BaseModel):
    id: uuid.UUID


class MyCloudEvent(CloudEvent, Generic[T]):
    data: T


if __name__ == "__main__":
    obj = MyEntity(id=uuid.uuid4())

    evt = MyCloudEvent(
        id=str(uuid.uuid4()),
        source="urn:source",
        subject="subject",
        data=obj,
        event_type="MyEntity",
        type="xyz",
    )
    # This works:
    # print(evt.model_dump(mode="python"))

    # But this does not:
    print(evt.model_dump(mode="json"))

Stack trace snippet:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/hans/Library/Application Support/JetBrains/PyCharm2023.2/scratches/scratch_42.py", line 30, in <module>
    print(evt.model_dump(mode="json"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hans/Library/Caches/pypoetry/virtualenvs/cvPU66vz-py3.11/lib/python3.11/site-packages/pydantic/main.py", line 308, in model_dump
    return self.__pydantic_serializer__.to_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Error calling function `_ce_json_dumps`: TypeError: Object of type MyEntity is not JSON serializable

[hozn]

We're using fastapi, so the workaround I'm using for the above nested-object serialization issue is:

from fastapi.encoders import jsonable_encoder

jsonable_encoder(event.model_dump(mode="python"))

Which works ... but this used to work without these extra hoops in pydantic v1 (and older cloudevents lib).

[alex-schaaf]

We also just ran into the same problem with pydantic v2 and cloudevents 1.11.0. We want to serialize the CloudEvent pydantic model (with a pydantic model as the data) to json before sending it out, so that we ensure that StrEnum values are properly converted to their values, but it crashes with the same error message:

event.model_dump_json()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/*/Library/Caches/pypoetry/virtualenvs/*/lib/python3.12/site-packages/pydantic/main.py", line 441, in model_dump_json
    return self.__pydantic_serializer__.to_json(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Error calling function `_ce_json_dumps`: TypeError: Object of type Payload is not JSON serializable

We now did a dirty workaround of first serializing the event data to json, then loading it back into a Python dictionary and the processing it further, before we emit our event.

[black-snow]

We now did a dirty workaround of first serializing the event data to json, then loading it back into a Python dictionary and the processing it further, before we emit our event.

Which is kinda silly because _ce_json_dumps basically does the same thing. So in the end the object gets json encoded and decoded several times, which is quite wasteful (and potentially prone to the bad kind of bugs).