C++ Demangler Rewrite

[b1llow]

This is the 3nd attempt to rewrite old C++ demangler GPL code.

New features:

C++20 name mangling support

Resources :

Old PR:

• Rewrite Of C++ Demanglers #69
• C++ Demangler GPL Code Rewrite #75

[notxvilka]

I know it would be a lot of work, but I recommend splitting tests semantically rather than arbitrarily. Group them by the feature, for example. In the long run it will be much easier to add new tests and improve old ones whenever necessary.

[brightprogrammer]

Hi!

Sorry for not being active for a long time. Last few (4-5) months have been a roller coaster ride for me. Thanks for picking up the PR. I just couldn't bring myself back at it again.

During my process of writing this, I modified the grammar so it's compatible with the parsing method here. I see that I didn't keep the grammar file here also I don't have access to the system on which it is on at the moment. I'll get it soon and make it available here so it's easier to follow the decisions made.

Sorry again 🙏 for keeping in the dark. RizinOrg is a lovely and welcoming community.

[brightprogrammer]

Attachments

4-final-almost-gnf-grammar.txt

The above file contains the final grammar that I deduced after making multiple transformations to the original grammar. The following file contains all files that I created during the transformations iteratively. Most probably if you ever decide to use the grammar files, you'll mostly use the above, but if you need any answer to "why" then following zip file may help.

steps-grammars.zip

How To Read The Grammar Files?

Rules are written like this

[.nested-name] <-- rule section start and anything that follows before any other rule starts is this rule's alternative
               <-- each rule alternative is represented in a single line
N [{{CV-qualifiers}}] [{ref-qualifier}] {prefix} {ctor-name} E
N [{{CV-qualifiers}}] [{ref-qualifier}] {prefix} {dtor-name} E
N [{{CV-qualifiers}}] [{ref-qualifier}] {prefix} {unqualified-name} E
N [{{CV-qualifiers}}] [{ref-qualifier}] {template-prefix} {ctor-name} {template-args} E
N [{{CV-qualifiers}}] [{ref-qualifier}] {template-prefix} {dtor-name} {template-args} E
N [{{CV-qualifiers}}] [{ref-qualifier}] {template-prefix} {unqualified-name} {template-args} E

• A rule is referenced like {rule} and
• A rule that may result in an empty string is represented like {{rule}}
• If a rule is optional then it's enclosed in [..] big brackets.

Derivation Process

This is the general GNF derivation process.

• I started with original grammar and prefixed all rules that surely start with a terminal with a . (dot). I called this a dot-rule.
• Then looked for rules that start with a dot-rule. This meant that this rule will eventually always start with a terminal as well and then marked this rule with a . (dot) prefix as well, meaning this is a dot-rule as well (inductively)
• Some rules that look very long and complicated is because of recursive application of this process.

This was done manually but I checked it manually more than hundred times so I believe this should be correct.

[b1llow]

@brightprogrammer I feel that implementing such a demangler using the C macro makes source-level debugging practically impossible. What was the original reasoning behind this approach?

[brightprogrammer]

Code size reduced by a lot and I could compare code with grammar easily. Debugging does become harder, that is one of the reasons it too so much time to even get here.

[brightprogrammer]

@b1llow The trace graph really helps here though. I found debugging using it quite helpful. Also because of the degeneracy of most of the code because of use of a select set of macros, only debugging of grammar rules is required, and some context information like substitutions and templates