Tag "productions" to allow using VSG as a VHDL parser#1309
Open
jfcliche wants to merge 11 commits intojeremiah-c-leary:masterfrom
Open
Tag "productions" to allow using VSG as a VHDL parser#1309jfcliche wants to merge 11 commits intojeremiah-c-leary:masterfrom
jfcliche wants to merge 11 commits intojeremiah-c-leary:masterfrom
Conversation
added 8 commits
November 2, 2024 21:10
…ng point of the productions
…ing them, now including production enter/leave lists. Refactor code to make code changes more managable
added 2 commits
December 17, 2024 23:19
…oken attributes such as production tags are preserved during parsing.
Author
|
Here is a updated PR, which was merged with the latest master (5845f5) . I had to re-introduce the systematic use of replace_token() to reassign tokens in order to preserve the token attributes such as production tags. Without that the production tags are lost in the post-token assignment processing and my VHDL parser not work anymore. tox generates lots of Pluggy errors, and I got a segfault at some point, but all tests eventually passed in the end. JF |
…ttributes are lost, including production tags. Moved replace_token to utils.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
I was looking for a pure-python VHDL parser to serve as a back-end for a Sphinx domain that would allow extracting documentation straight from the VHDL source code, like we do with Python docstrings.
After looking around and testing various tools, VSG turned out the only VHDL processor that was fast and robust enough to process my code base. The only thing that was missing was information on the hierarchy of the VHDL elements (entities, ports, etc.), also known as "productions". VSG knows about this hierarchy when it "classfies" the tokens, but it does not save the information. (See Discussion #1131)
This pull requests adds a minimally-intrusive mechanism to tag the productions. A
@utils.tagged_productiondecorator, when applied to aclassifyfunction, adds the name of the current production in theenter_prodlist of the token at the beginning of the classify code, and also adds the name of that production to theleave_prodlist of the token when the classify function exits. It is then trivial to reconstruct the hierarchy of the whole code using these tags.This pull requests adds a tagging function and decoration in
vhdlFile/utils.py, the two abovementioned lists in theitemclass (parser.py), and a few classify functions have been decorated. This is currently sufficient for me to extract the entity interface information, although so much more could be done. The code has been modified to make sure thatconvert_tois always used when we replace a token by another, otherwise we lose our production tagging information.This pull request also simplifies the
post_token_assignments()function invhdlFile.pyby removing some code repetitions. Although I did that work a few months ago, I am submitting it just anow and I just realised that another PR #1301 was submitted a few days ago to the same effect. That PR seems to implement changes that I also had in mind but did not dare do yet. I'd be happy to update my PR to harmonize it with PR #1301 if that helps.The code passes the tox tests.
This is my first public PR ever. Let me know if other information is required.
JF