Request to provide cFS/cFE XTCE Files

[jbell-astrobotic]

Hi! I noticed applications have started to provide COSMOS and EDS files, but no XTCE files. Any particular reason these are omitted? I imagine many people would enjoy having XTCE files provided out of the box with releases to use with Yamcs and OpenMCT.

[lorenzo-gomez-windhover]

@jbell-astrobotic If you are interested in XTCE files, our team wrote a tool that can auto-generate them for you. https://github.com/WindhoverLabs/auto-yamcs?tab=readme-ov-file#how_to_use_it_quick

[keegan-moore]

Thanks for calling this out!

[mbenson1]

I know at least one person at GSFC was using an early prototype of this tool several years ago, called Explain. We have since designed and written a suite of new tools dedicated to this, and stopped maintaining Explain. As you know, most implementations of CI and TO just copy memory in and out of the transmission media rather than serialize/deserialize to make the most out of limited CPU and bandwidth. Therefore the bit pattern is almost totally dependent on the target and tool chain ABI. You can reverse engineer and predict the compiler behavior to prepackage XTCE but it will always be sensitive to tool chain differences and code changes.

We approached this differently by scraping the bit patterns directly from the as-built binary in the same way that debuggers do, rather than scrape source code and predict compiler behavior. The tool suite then generates correct XTCE that matches both your tool chain behavior and your code changes. Most programs have a team dedicated to generating and maintaining the uplink and downlink definitions. We have been using this tool for 6 years now without a dedicated team. Developers and ops build their own displays, but nobody writes uplink or downlink definitions. Since we don't touch the XTCE, we add telemetry and commands to the system as fast as we can add the new code to the flight software.

It is a collection of tools chained together so its not married to the XTCE format. The pipeline starts by extracting data definitions from the as-built binary, populating a database. We have some other tools that add additional data to the database, like NASA's CCDD but it contains everything that can be extracted from the binary which is significantly more than just telemetry and command definitions. Then the XTCE generation script then parses the database and generates the XTCE. XTCE is a loose standard that, in our experience, is never fully implemented by all ground systems, so there is often a separate script for each "flavor" of XTCE. The one packaged with "auto-yamcs" is the YAMCS flavor. We have also created scripts that generate input files for non-XTCE based ground systems.

This is also not exclusive to CFS. We have used this as a component to a simulation so it would automatically translate between running the flight software on a Linux PC and VxWorks on a PPC. Rather than generating XTCE to serialize and deserialize, it generated memory translation maps that allowed another tool to automatically reconfigure incoming and outgoing messages to match the recipients ABI without requiring the sim developers to know or even care that the other end had a radically different memory layout.

We released it open source on Github and it is actively maintained.

[jbell-astrobotic]

This is super neat - thanks guys! I'll take a look at this when I can find some spare time. Interesting concept - seems like you all have chosen the route of C header (or at least compiled binary) -> XTCE (and other formats) whereas a project I'm on we've chosen to go the reverse route of XTCE -> auto-generated C headers.

[mbenson1]

Not the header. It doesn't read source code at all. Source code doesn't always capture endianness and invisible padding due to alignment, bit packing, optimization, variable data type bit length, invisible attributes like vtable address, etc... But the as-built doesn't lie. If you mean that the data definitions have the same structure and element names as the header file, this is mostly correct. The only time it differs is when it finds invisible padding, or, God forbid, anonymous attributes. When this happens, we give it a name that clearly denotes that it is implied. We do this purposely so we have the option of actually seeing what might be in that "inaccessible" memory in case we suspect that flight software has a bug and is overwriting memory incorrectly.

For XTCE and other ground system input generation, it still needs to know what structure is associated with which message ID, or however you identify your messages. If the message IDs are assigned at runtime, this data can be supplied by EDS. We've also generated the message ID files from EDS, and extracted the message ID to structure association directly from the as-built.

Like the CCDD, once the data is in the database, any data can be generated, not just XTCE files. We have used it to generate COSMOS input files. Its just we use YAMCS internally which supports XTCE directly, so we make that preconfigured pipeline readily available.

We generate a lot of our header files too. We've used this tool to audit builds to ensure that the macro values are set correctly from the generated files. It sounds like it should be unnecessary but this uncovered an flight software build system bug where the same macro name was accidentally used in multiple places. The compiler didn't catch it as a redefinition because of a convoluted series of inclusion paths resulting in a header file accidentally being included by certain .c files and another being included by other .c files. It went unnoticed for a long time because both macros had the same value, so it was caught before it became a problem. We've used the tool to not only double check that it was built with the correct macro values, but that it got the values from the correct header file.

Lastly, we also use the tool to determine what build products actually go into each application and table. We've had to deliver build projects to customers that include some prebuilt proprietary applications without source code, but still allow them to add their own applications, modify the open source applications, and generate new tables. Our main project has many targets and configurations and our build system supports a hierarchical configuration structure, but partial deliveries can get complicated. To automate the project creation, we select which products the customer will need to rebuild, review the list of files needed to build that product, and approve or reject it for release.

[mbenson1]

I made my email address public in case you want to further information.

[keegan-moore]

@jbell-astrobotic - The hope is that EDS can eventually be used to auto-generate XTCE definitions, as well as other command/telemetry definitions (like our COSMOS packet defs). This is mostly a limit of time/resources, but we're working on improving this over time.

[jphickey]

To further explain that -- XTCE is really targeted with ground systems and there is information needed to create XTCE that is not really know/relevant at the embedded FSW or device level. This is really where EDS comes from, it is a cousin to XTCE (and in fact shares some schema elements) but targeted toward documenting embedded devices rather than ground systems. The information in EDS can absolutely be used to generate XTCE files but you will have to supplement it with additional data to make it into a proper ground system DB.

The EDS XML schema is standardized through CCSDS and the intent is that it can be converted into whatever proprietary or tool-specific language you need, such as the CCDD JSON, COSMOS, DDS/IDL, ITOS or whatever else. So rather than forcing users into a particular tool and/or cherrypicking which tools we support, this is a way to basically support all of them and allow end-users to choose whatever product they prefer, but still have the benefits of a reliable messaging interface that is not at the mercy of your C compiler and processor architecture

[jbell-astrobotic]

For sure, totally understand EDS is used among the industry as a ground truth to auto-generate other file formats. I'm trying to align what I guess may be unfounded assumptions on how widespread Yamcs use is. e.g. Viper uses Yamcs and OpenMCT so I thought the general support for Yamcs XTCE definitions may be a bit more common - so was confused to see support for COSMOS before Yamcs and just in general I heard the word 'COSMOS' multiple times during the symposium this week and no mention of Yamcs or OpenMCT. I may challenge the 'cherrypicking which tools we support' comment when COSMOS cmd/tlm is the only schema provided in a release aside from EDS!

[jphickey]

Yes, there is a lot of talk about COSMOS, it is gaining in popularity. But to clarify, we do not "directly" provide a COSMOS DB, what we have provided is a script that uses EDS information to generate a COSMOS DB. This is not the only output product though, we also generate a JSON DB compatible with CTF/CCDD, and an IDL file that is intended to work with DDS, along with generic bindings for Python and Lua and the ability to generically translate CMD/TLM objects to and from JSON. And the framework allows one to write their own conversion to any other sort of format. The problem is knowing what would be the most useful, and COSMOS seems to be a useful thing to a lot of users, hence why it was in the set.

That being said, I do find YAMCS interesting as a way to reliably create an EDS from an existing app that does not have one. I believe there could be a very useful an important role for a tool like that, but I am only learning of its existence this week.