Converting Entry to JSON does not include field values for multi-nested resources

[kfleitman06]

I am currently using the Contentful client in Python within one of our APIs to get entries matching a query and return the entries as a response using ujson.

When we get an entry using client.entries(), we are able to see all linked resources are hydrated properly in each entry's _fields property, however Python does not include properties for objects that are prefixed with _ when serializing to json using ujson or json.

https://github.com/contentful/contentful.py/blob/master/contentful/resource.py#L113

This means that when we return the response of a list of entries with multiple nested resources from our API, we are unable to use the nested entries due to not having the hydrated fields resources, despite them having originally been hydrated

Roughly, for example:

When looking at the only entry in a list in the response from client.entries(), where the content is multiple levels of nested references

a_entry.fields() = {
  title: "Entry A",
  content: [
    B_Entry: {
      sys: {...},
      _fields: {
         title: "Entry B",
         content: [
           D_Entry: {
               sys: {...},
              _fields: { 
                  title: "Entry D",
                   ...
              }
           }
        ]
      }
    },
    C_Entry: {
      sys: {...},
      _fields: {
        title: "Entry C",
        ....  
        }
     },
  ]
}

serializes to json:

 {
  title: "Entry A",
  content: [
  {
   "sys": {...},
  },
  {
   "sys": {...},
  }
]

}

Where sys only holds references to the link IDs, and all actual fields within _fields are lost that aren't in the top-level entry

Is there something that I am missing here to be able to easily have the field values included as JSON?

I understand that we can set the value of each entry to a new key by doing [entry].fields(), but when we have many levels of nested resources included, it feels complex to have to go back through each level to set a key like field to be the value of _field after it was already built

[rubydog]

Hi @kfleitman06, thanks for reporting the issue.

I will take a look at it and get back to you if I have any questions.

[rubydog]

Hi @kfleitman06, can you please share the code snippets to produce the results mentioned above?

Thanks

[kfleitman06]

hi @rubydog, thanks for taking a look

I'm able to produce these results with the following code

import ujson

import contentful

SPACE_ID = "<REDACTED>"
ACCESS_TOKEN = "<REDACTED>"


def main():
    client = contentful.Client(
        space_id=SPACE_ID,
        access_token=ACCESS_TOKEN,
        content_type_cache=True,
        reuse_entries=True,
    )

    entries = client.entries(
        {
            "content_type": "<REDACTED>",
            "fields.pageType": "features",
            "fields.slug": "<REDACTED>",
            "include": 10,
        }
    )
    if not entries:
        raise Exception
    
    entry = entries[0]

    fields = entry.fields()
    print(ujson.dumps(fields))  # this value is returned as a FastAPI response, rather than used further within this codebase


if __name__ == '__main__':
    main()

Within fields, we have fields defined such as content (a list of entries) and hero (a single referenced entry, containing fields that also have referenced entries) . The nested referenced entries mentioned prior to doing ujson.dumps() show the full resolved value under _fields, however _fields is then lost on each after ujson.dumps(), resulting in only having the link/entry ID available for any information, under raw

We currently have a support ticket open with Contentful, if you are able to access that. I can provide the redacted args from the query we are making for this example if that is helpful

[rubydog]

@kfleitman06 are you using the latest version of the library? If not, can you please update to the latest version and let me know if the problem persists?

[gcampes]

Just realised you were waiting for a response here 😢

Yep we were on the latest version

I believe we ended up work arounding this by recursively accessing all referenced properties before sending the parent entry over as a response.

Would be good to not need to do this step though, any thoughts on this or anything else you need from our side to help on this one?