Visualization of the openage codebase

[ivanmilevtues]

Hey all, I am building a visualizer of codebases.
You can check it out here: https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/openage/on_boarding.md

The goal is for a new person to be able to get up-to-speed and start working on the project as soon as possible. I.e. usually you start small so, we think that context is very important, this is why we have these high-level diagrams - you get context of what the main components are and how they interact with each other. Then you can drill down into what you find interesting and now can contribute with the context of how the whole thing integrates.

Lmk, what do you think of the diagram first documentation! Any feedback is more than welcome!

Full disclosure: we're trying to turn this into a startup, but we're still in a very early stage and figuring out what will actually be useful for people. Also as many people brought that the diagrams will become outdated fast, we've introduced free github action!

[heinezen]

Hello,

thanks for sharing! I have written most of the code your tool analyzed, so I can certainly give some feedback. Generally I like the idea of your approach, although I can still see some of the limitations in complex code bases such as ours. Some things I noticed:

• You onboarding overview completely misses the C++ part of our code. I think that's because you focus on Python projects right now, but it was a bit surprising to see.
• I think your visualizer can roughly spot the important code parts, but it also focus on unimportant parts like the DRS file parsing, which is not that relevant for the code compared to other file formats that are parsed (Like SLP and DAT).
• The selection of related classes/methods is also often not that great, e.g. for Core Infrastructure & Utilities it lists a bunch of random stuff that's not always part of utilities (such as the API export).
• Your LLM hallzinated stuff like the VersionDetector which does not exist as a class.
• The diagrams are a bit weird in my impression because their purpose is not really clear to me. Are they supposed to visualize the flow of the code or are more about showing relationships? I guess it's the latter, but I'm not really sure. I think they could be more helpful if they were more integrated with the text.

Some thing I also noticed is that your generated onboarding docs could have the same problems that a lot of hand-made documentation also has. Specifically, I mean documentation that lists of bunch of classes and says what they do, but doesn't really describe the overall patterns, structure, and design principles of the systems. That is something I unfortunately rarely see in docs, although it's pretty important to maintain the code. I don't know if your visualizer can provide that, but it's probably a good idea to look into it.

[ivanmilevtues]

Hi @heinezen, thanks for taking the time and looking into it, I really value it!

I will address your points in the same order:

1. Yes we have skipped over the C++ part of the codebase, as we support just Python. The approach we have stems in Static Analysis so that we can keep LLM's under control.
2. Your second point is amazing, the interesting parts of code are very interesting concept. We have fiddled arround it, but it is (in my opuinion) quite hard to come up with a heuristic on what is important and what not, especially in static analysis. I.e. with dynamic analysis one can find the hot spots and it could be that these are the important parts 🤔. I think another thing that we can make use of is just existing doc, however we try to avoid it as it can be very well that there is no existing docs, or they are way too outdated (which is what we are trying to solve).
3. Thanks for the feedback on the related code, I will definetly take a look into it, as I believe it is one of the most important things to get right - shows that what we do actually makes sense and is not an LLM slop.
4. I will also take a look into the hallucination, the whole point of using static analysis is to avoid exactly this kind of situations!
5. What we want to get out of these diagrams is a high-level understanding/context for a new person of what the main components of the source code are and how they interact with each other. So that is why it is a bit of a mix from both structure and flow - as in some cases the main components are structurally intergrated (inheritence, composition, etc), but also sometimes it is behavioral. So we kind of mix the two in the same place.

Specifically, I mean documentation that lists of bunch of classes and says what they do, but doesn't really describe the overall patterns, structure, and design principles of the systems.
This is very interesting, you are the second person who notes it. I am also curious to see what we can do this in this space, I just wonder at what level it will kick in. I.e. at the highest level of abstraction I suppose it is the archtectural pattern (MVC, MVVM, etc.)
And then the deeper you go the more Gang of four (classic OOP patterns) will start showing up, just unsure how to integrate this in our view UI wise. Would be interesting to hear if you have an idea of how you'd like to see!

[ivanmilevtues]

Hey a quick update on our side, we've just released our engine as an open-source repo: https://github.com/CodeBoarding/CodeBoarding

@heinezen also to touch on the CPP point, we are now implementing LSP (Language server support) for our static analysis part so hopefully we will support CPP very soon i will update you when that happens!