Investigate Data Format Description Language (DFDL)

[nichtich]

This is a rather broad issue, not a specific feature request, as I did not find any mention of Data Format Description Language (DFDL) in this repository.

I have not worked with DFDL but as far as I understand it, given a DFDL schema (see examples here), a DFDL processor should be able to parse data in this format. To allow fq parsing arbitrary formats with known DFDL schema it needs

• either an implementation of a DFDL processor in Go
• or a binding to an existing DFDL processor (probably Apache Daffodil) from fq
• or a compiler to create Go code from a DFDL schema

Sure, support of DFDL is far from trivial and I don't know if its worth the effort, given the current popularity DFDL, but in any case fq and DFDL are related enough to better join some efforts in the long run.

[wader]

Hey, looks interesting and it reminds me of kaitai. I've spent some time on a kaitai interpreter prototype for fq which looks promising but there is quite a lot of work left on it, but i think DFDL and kaitai would be implemented in similar ways.

The main difficulties i can see is:

• Implement the DFDL expression language
• Is the java regular expression dialect compatible enough or translatable to golang stdlib regex?
• How complex is the type system? for kaitai i run into some difficulties understanding how types are resolved etc

Is this something you would like to work on? i'm thinking maybe i should refactor out some things from my kaitai branch that can probably be shared.

[wader]

BTW there is a related issue #627 about runtime decoding

[nichtich]

Is this something you would like to work on?

No, just wanted to create a link betwee DFDL and fq but missed the obvious link in README.md as I only searched for the acronym "DFDL". Seems like nobody has mentioned fq and neither Kaitai in the Daffodil issue tracker and mailing lists.

If you happen to implement Kaitai struct anyway, mapping DFDL to Kaitai seems a better approach because such mapping could also be used independently from fq (kaitai-io/kaitai_struct#441). I guess it won't support all features of DFDL but enough to get the communities in touch.

[wader]

If you happen to implement Kaitai struct anyway, mapping DFDL to Kaitai seems a better approach because such mapping could also be used independently from fq (kaitai-io/kaitai_struct#441). I guess it won't support all features of DFDL but enough to get the communities in touch.

By mapping you mean add DFDL output support to the kaitai projects? yeap that sounds resonable