Implementing new language backends

[alknemeyer]

Hello,

Is there a guide describing how to implement a new language backend for CycloneDDS? I understand there are two parts to this: IDL compilation, and runtime usage of the dds library. A *_desc topic descriptor needs to be generated too.

I can't find any info on this from the CycloneDDS docs, so I imagine this info is only available from the DDS specs. Otherwise, IDL compilation and library usage can be figured out by looking at the C++ and Python implementations (although it's hard to say if I'm missing something important) but the "magic" bits generated for type information remain a mystery.

I'm not asking you to document all that, but a short note about advised steps/useful resources would be lovely :)

Thanks!

[eboasson]

Hi @alknemeyer, I'm afraid there is no such guide. In addition to the Python and C++ ones, there is also a rust binding from @sjames, that you might useful, too. In any case, please just continue asking questions.

The run-time usage of the dds library is pretty obvious and the only complicated thing really is just the type support. There are two routes (or maybe two-and-a-half, now that we can dynamically construct types using the C API):

• translate data types between your language mapping and C;
• do a everything custom for the language.

The former is relatively easy especially because you don't have to deal with the vagaries of XTypes serialization formats. Then you just need to provide a backend to IDLC that spits out your type definitions and some conversion routines that copy to/from the C API representation, and in the mapping to the Cyclone core library you simply use dds_create_topic and the ..._desc you get out of the C backend of IDLC. The downside is obviously the cost of copying.

The latter gives you full control, but you then you have to write your own "type support" (a.k.a. sertype / serdata in Cyclone for historical reasons), and with that you can then use dds_create_topic_sertype like the other language bindings do. For figuring out how to write one you probably want to refer to #830 because it contains a wealth of background information. Some minor details have a changed a bit with the introduction of XTypes, but on all the important points it is still correct.

There are of course some strange incantations involved that are caused by using an old interface that was exposed before it was ready, &c., but the real pain here is having to write the serializers. Going down this path, my advice would be to first do the original CDR because unlike the new XCDR2 that is totally straightforward. For backwards compatibility reasons, it is the default in Cyclone for all @final topics using types that a non-XTypes implementation can handle.

The type information part of XTypes is not strictly necessary, thanks to backwards compatibility: if it isn't present, the reader/writer matching looks at the type name instead of the type definition. On the other hand, those magic bits all come rolling out of the IDL compiler (or can be obtained dynamically) and you really only need to make it available to the library so it can integrate it in the discovery information. You might save yourself a lot of time by also having a look at the RMW layer and the PR also linked above.

Hopefully this helps you get started, and again, please do keep asking questions. It'd be great to see more language bindings!

[alknemeyer]

Hi, thank you for the detailed reply! And sorry for taking so long to reply.

We implemented a code generator by leveraging idlc's plugin system (ie by compiling a libcycloneddsidl${LANG} shared object and calling idlc -l ${LANG} data.idl). The runtime was straightforward usage of the C API, as you mentioned. We didn't implement the type support stuff (CDR and so on) and instead ran IDLC again to generate C code, linking in the appropriate ..._desc variables.

I'll update this thread with some comments on things I felt weren't so clear, which I would have appreciated when starting. After some feedback/corrections from you maybe we could write a doc for this, even if it's largely links to other issues. I'm going on holiday tomorrow so it'll be sometime next week :)

[eboasson]

Hi @alknemeyer that's great news 😀 The feedback is something I look forward to and if you'll indeed find some time for adding a bit of documentation for this, that would be great.