Adding DotNotation Support and Widget Framework to existing cli (#1495) by LeighFinegold · Pull Request #1496 · finos/architecture-as-code

LeighFinegold · 2025-08-07T02:36:11Z

Overview

This PR introduces the foundational Widget Framework and Dot Notation capabilities for CALM templating, putting them structurally in the right place:

Dot Notation: Available for both calm template and calm docify commands
Widget Framework: Specifically designed for calm docify command

Related to: CALM Docify CLI v1.0.0 MVP #1452

Key Changes

Dot Notation Support (for calm template & calm docify)

✅ Template preprocessor for natural path syntax (nodes['unique-id=="service"'])
✅ Support for bracket notation, filters, and mixed expressions
✅ Handlebars keyword protection

Widget Framework (for calm docify only)

✅ Standalone widget system with registry, engine, and renderer
✅ Type-safe widget validation and error handling
✅ Three foundational widgets: table, list, json-viewer

Template Engine Integration

✅ Naturally integrates with new canonical model and dereferencing (Dereferencing and Canonical Modeling Strategy in CALM #1471)
✅ Path extraction and processing pipeline

What's Working

Dot Notation (Both Commands)

{{!-- Natural dot notation --}}
{{ architecture.nodes['service-unique-id'].description }}

{{!-- Advanced filtering --}}
{{ nodes['node-type=="service"'] }}

{{!-- Mixed notation --}}
{{ relationships['type=="connect"'].nodes }}

Widgets (calm docify Only)

{{!-- Widget rendering with dot notation paths --}}
{{table nodes['node-type=="service"'] columns="name,port"}}
{{list relationships['type=="connect"'].nodes}}
{{json-viewer architecture.metadata}}

Future Enhancements

Widgets to be Enhanced

Expand validation and options for existing widgets

Dot Notation to be Enhanced

Extended capabilities leveraging JSONPath internally
Cleanup of core code with better logging and aid to end users

CALM-Specific Widgets (Next Phase)

flow-sequence - Interactive sequence diagrams
c4-architecture - System architecture views
related-nodes - Node relationship visualization
Cleanup of current template bundle for docify website mode

Known TODOs

The following has been identified but should not block this initial PR. This will allow other maintainers to dive in and contribute.

Helper Function Conflicts: Resolve naming between widget helpers and the default transformer --tempalte and template-dir are using under covers.
Architecture Refactoring: Move docifier/graphing packages to widget framework
Error Message Enhancement: More detailed widget validation feedback
Handlebars Isolation: Dependency injection for better testability
Documentation: Documentation to be added to main docs website prior for v1.0 release.

Testing

✅ Widget validation and rendering tests
✅ Integration tests with real CALM data
✅ Error handling and edge case coverage

Migration

✅ Backward compatible with existing calm template and calm docify workflows
✅ No breaking changes to current template bundles
✅ Gradual adoption path for new features

(@markscott-ms, @aidanm3341 , @willosborne , @Thels , @rocketstack-matt )

…rk (finos#1495)

LeighFinegold · 2025-08-07T07:02:40Z

@markscott-ms, @aidanm3341 , @willosborne , @Thels - as discussed yesterday, pushing what have so more can contribute in parallel

markscott-ms · 2025-08-07T08:29:42Z

@LeighFinegold could you add issues for your known todos so they are more visible for others to assign and pick up?

rocketstack-matt

I've tested this locally and it is frankly incredibly cool . . .

# Example table
{{table nodes['load-balancer']}}

# Example list
{{list relationships property="unique-id"}}

# Example JSON View
{{json-viewer nodes}}

produces

One big issue I came across we should fix was when initially passing the output file as in the same directory I was working in (which happened to be the root of the repo) . . . it deleted everything in the directory without warning 😢

I think this is existing behavior, but I wasn't expecting it because I gave an output file not just a directory. Either way, I think we should at the very least add a warning that the output dir will be cleaned and prompt for confirmation. In the case of passing an output file (as opposed to a directory) I don't think it should clean out the directory at all.

LeighFinegold · 2025-08-07T09:08:59Z

@LeighFinegold could you add issues for your known todos so they are more visible for others to assign and pick up?

Follow up issues will be raised over the next few days and will mark related items as good first issues for cli developers

LeighFinegold · 2025-08-07T09:11:32Z

I've tested this locally and it is frankly incredibly cool . . .
# Example table
{{table nodes['load-balancer']}}

# Example list
{{list relationships property="unique-id"}}

# Example JSON View
{{json-viewer nodes}}
produces
One big issue I came across we should fix was when initially passing the output file as in the same directory I was working in (which happened to be the root of the repo) . . . it deleted everything in the directory without warning 😢
I think this is existing behavior, but I wasn't expecting it because I gave an output file not just a directory. Either way, I think we should at the very least add a warning that the output dir will be cleaned and prompt for confirmation. In the case of passing an output file (as opposed to a directory) I don't think it should clean out the directory at all.

It's existing behaviour to do a clean step at the beginning, but I agree a single file should only delete the file and not the whole directory when using the --template option. Will raise an issue describing the problem

Raised #1505

markscott-ms · 2025-08-07T08:56:39Z

calm-widgets/src/widget-engine.ts

+    }
+
+    registerDefaultWidgets() {
+        const widgets: { widget: CalmWidget<unknown, object, unknown>, folder: string }[] = [


nitpick: have this list in alphabetical order of widget name?

Raised #1501

calm-widgets/src/widget-helpers.spec.ts

markscott-ms · 2025-08-07T09:03:31Z

calm-widgets/src/widget-registry.ts

+import fs from 'fs';
+import path from 'path';
+import Handlebars from 'handlebars';
+import { CalmWidget} from './types';


nitpick: inconsistent spacing within braces

This should be handled by some automatic linting we run pre-PR.

This won't be the only case of this in the code-base.

The ideal is the IDE would auto correct those. #1396 has been raised to run strict mode. Raised #1502.

I can address this one instance, but will not fix the lint rule now as there are likely lots of other instances and occurances of hte same. This should be addressed in a future office hour.

I addressed the minor comment all the same on last commit

markscott-ms · 2025-08-07T09:15:25Z

calm-widgets/src/widgets/list/list-template.html

@@ -0,0 +1,29 @@
+{{#if ordered}}


There is a lot of duplicate code in this file - the only things that change are ol <=> ul. Can/should the ordered if/else be around the opening and closing tags only?

We could delegate the tag to the viewmodel and just print that. Welcome an issue or PR post merge if we want to address this.

Given the urgency of needing this in - can you raise an issue for this and we can merge without?

markscott-ms · 2025-08-07T09:19:20Z

cli/src/cli.e2e.spec.ts

    afterAll(() => {
        if (tempDir) {
-            fs.rmSync(tempDir, { recursive: true, force: true });
+            //rs.rmSync(tempDir, { recursive: true, force: true });


Is this line of code meant to be commented out?

It has no impact to the build as ultimately everything is discarded in the end, but this is something that is done to view the outputs when there is an issue in more detail. It occurs on the cli.e2e.spec.ts tests because these are the tests where we on doing a full simulation of using the npm module and therefore harder to debug through problems i.e. this is done so i can debug using the index.js.

Testing approach has been brought up in a few office hours. Will follow up with @willosborne and see if we can raise an epic around testing practices we are looking to adopt and cleanup.

I addressed the minor comment though on last commit

Deletion of directory is existing behavior and will be addressed in a subsequent PR

Thels · 2025-08-07T11:27:06Z

I think that this is a big PR and @LeighFinegold has put a considerable amount of his own personal time into getting this into the state it's in.

I've pulled down the PR and it's behaving, looks like @rocketstack-matt also has done it based on the conversations above. I think we should approve and then refactor/improve as we go on instead of trying to make it perfect from the get-go.

@markscott-ms - you good with this too?

rocketstack-matt · 2025-08-07T11:47:14Z

I think that this is a big PR and @LeighFinegold has put a considerable amount of his own personal time into getting this into the state it's in.

I've pulled down the PR and it's behaving, looks like @rocketstack-matt also has done it based on the conversations above. I think we should approve and then refactor/improve as we go on instead of trying to make it perfect from the get-go.

@markscott-ms - you good with this too?

Agreed - I've approved, @markscott-ms if you're happy with he issues being raised post merge lets move ahead.

markscott-ms · 2025-08-07T11:47:36Z

@Thels I agree with merging this in to allow divide and conquer on anything spotted.

Functionally and conceptually this looks great.

Adding Structurally foundation DotNotation Support and Widget Framewo…

d797ee6

…rk (finos#1495)

LeighFinegold requested review from a team, Thels, aidanm3341, grahampacker-ms, jpgough-ms, lbulanti-ms, markscott-ms, rocketstack-matt and willosborne as code owners August 7, 2025 02:36

github-actions bot added cli shared config labels Aug 7, 2025

LeighFinegold marked this pull request as draft August 7, 2025 02:51

LeighFinegold force-pushed the calm-widgets branch from bf25a57 to 401ebac Compare August 7, 2025 06:36

Add a build for calm widgets (finos#1495)

210ccc9

LeighFinegold force-pushed the calm-widgets branch from 401ebac to 210ccc9 Compare August 7, 2025 06:54

LeighFinegold marked this pull request as ready for review August 7, 2025 07:02

rocketstack-matt previously requested changes Aug 7, 2025

View reviewed changes

Merge branch 'main' into calm-widgets

53a3128

markscott-ms requested changes Aug 7, 2025

View reviewed changes

Merge branch 'main' into calm-widgets

121c984

LeighFinegold mentioned this pull request Aug 7, 2025

Enforce Alphabetical Ordering for Widget Helpers and Widgets #1501

Open

6 tasks

Fixing minor comments

748d4e5

markscott-ms approved these changes Aug 7, 2025

View reviewed changes

LeighFinegold merged commit 51da13f into finos:main Aug 7, 2025
13 checks passed

Conversation

LeighFinegold commented Aug 7, 2025

Overview

Key Changes

Dot Notation Support (for calm template & calm docify)

Widget Framework (for calm docify only)

Template Engine Integration

What's Working

Dot Notation (Both Commands)

Widgets (calm docify Only)

Future Enhancements

Widgets to be Enhanced

Dot Notation to be Enhanced

CALM-Specific Widgets (Next Phase)

Known TODOs

Testing

Migration

Uh oh!

LeighFinegold commented Aug 7, 2025

Uh oh!

markscott-ms commented Aug 7, 2025

Uh oh!

rocketstack-matt left a comment

Choose a reason for hiding this comment

Uh oh!

LeighFinegold commented Aug 7, 2025

Uh oh!

LeighFinegold commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Thels commented Aug 7, 2025

Uh oh!

rocketstack-matt commented Aug 7, 2025

Uh oh!

markscott-ms commented Aug 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

LeighFinegold commented Aug 7, 2025 •

edited

Loading