Bug: serializing/deserializing of JSONfields fails

[markdoerr]

Dear developers,

JSON de-serialization fails with the following Error:

python data_grpc_client.py
Traceback (most recent call last):
File "/my_data/data_grpc_client.py", line 8, in
for datum in stub.List(my_data.lara_data_pb2.DataListRequest()):
File ".../python3.9/site-packages/grpc/_channel.py", line 426, in next
return self._next()
File .../lib/python3.9/site-packages/grpc/_channel.py", line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Exception iterating responses: Failed to parse data_JSON field: expected string or bytes-like object."
debug_error_string = "{"created":"@1648155177.535827174","description":"Error received from peer ipv6:[::1]:50051","file":"src/core/lib/surface/call.cc","file_line":905,"grpc_message":"Exception iterating responses: Failed to parse data_JSON field: expected string or bytes-like object.","grpc_status":2}"

What is missing to support JSONField serialization / deserialization ?

To reproduce the Error:

I have a django (4.0.3) model with a JSON field:

# models.py

class Data(models.Model):
  data_id = models.AutoField(primary_key=True)
  data_JSON = models.JSONField(blank=True, null=True, help_text="JSON representation of the data")

My serializer.py looks like this:

from  my_data.models import Data
from django_grpc_framework import proto_serializers
from rest_framework import serializers
import my_data.my_data_pb2


class DataProtoSerializer(proto_serializers.ModelProtoSerializer):
    class Meta:
        model = Data
        proto_class = my_data.lara_data_pb2.Data
        fields = ['data_id', 'data_JSON']

The auto generated protofile file (I used the django framework proto generator) my_data.proto:

syntax = "proto3";

package my_data;

import "google/protobuf/empty.proto";

service DataController {
    rpc List(DataListRequest) returns (stream Data) {}
    rpc Create(Data) returns (Data) {}
    rpc Retrieve(DataRetrieveRequest) returns (Data) {}
    rpc Update(Data) returns (Data) {}
    rpc Destroy(Data) returns (google.protobuf.Empty) {}
}

message Data {
    int32 data_id = 1;
    string data_JSON = 2;
}

message DataListRequest {
}

message DataRetrieveRequest {
    int32 data_id = 1;
}

My service description - services.py :

from my_data.models import Data
from django_grpc_framework import generics
from my_data.serializers import DataProtoSerializer


class DataService(generics.ModelService):
    """
    gRPC service that allows users to be retrieved or updated.
    """
    queryset = Data.objects.all().order_by('data_id') 
    serializer_class = DataProtoSerializer

And finally the client: data_grpc_client.py:

import grpc
import my_data.my_data_pb2
import my_data.my_data_pb2_grpc


with grpc.insecure_channel('localhost:50051') as channel:
    stub = my_data.my_data_pb2_grpc.DataControllerStub(channel)
    for datum in stub.List(my_data.my_data_pb2.DataRequest()):
        print("datum:",datum.name, end='')

[AMontagu]

Hi

We are maintening: https://github.com/socotecio/django-socio-grpc/ that has a different way of generating proto using serializers and services. We are supporting JSONField and other complex fields.

If you want to try it and give us a feedback it would be great.