Skip to content

Diagram editor #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 176 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
176 commits
Select commit Hold shift + click to select a range
cd129ef
first commit
koonpeng May 16, 2025
1f02db5
fix section schema
koonpeng May 16, 2025
e273f86
update schemas
koonpeng May 16, 2025
35ae56e
load diagram json into reactflow nodes
koonpeng May 20, 2025
bb28f62
successfully display a diagram
koonpeng May 21, 2025
de5902f
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng May 21, 2025
0facc77
visualize other nodes
koonpeng May 21, 2025
cf67647
organize workspace
koonpeng May 26, 2025
6e5907d
add auto layout button
koonpeng May 26, 2025
79ae317
filter upload files
koonpeng May 26, 2025
fa5bd64
aria-label consistent with tooltip
koonpeng May 26, 2025
cf5a186
add operations
koonpeng May 26, 2025
6ec072f
rust crate to serve an embedded frontend
koonpeng May 27, 2025
8d5e4bb
cleanup
koonpeng May 28, 2025
3ff869c
add form to edit node operation; ux improvements
koonpeng May 28, 2025
d887a39
serve diagram editor from root to make url rewriting easier
koonpeng Jun 4, 2025
7c67496
update comment
koonpeng Jun 4, 2025
4bb6b74
fix uncaught error when base64 decode fails
koonpeng Jun 4, 2025
84fec77
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jun 9, 2025
a625906
support exporting diagram
koonpeng Jun 9, 2025
6a12ae2
wip syncing operation connection and react flow edges
koonpeng Jun 9, 2025
4214105
add edge data when loading diagram
koonpeng Jun 10, 2025
b2c282e
fix loading empty diagram; add copy share link button
koonpeng Jun 10, 2025
9d8255d
nextOperationToNodeId returns null for dispose
koonpeng Jun 10, 2025
8901ebb
use builtin dispose as default for new ops
koonpeng Jun 10, 2025
2780ce5
use small textfield for share link
koonpeng Jun 10, 2025
aac0121
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jun 11, 2025
086ecef
add buffer form
koonpeng Jun 11, 2025
b0ef111
add editing unzip edge
koonpeng Jun 11, 2025
a7f5edb
move edges to another directory
koonpeng Jun 12, 2025
2a27f05
refactor and add custom unzip edge component
koonpeng Jun 12, 2025
c5ecfba
use exhaustive check when possible
koonpeng Jun 12, 2025
a0109dd
add definitions for all edges
koonpeng Jun 12, 2025
4ca6718
add edit forms for other edges
koonpeng Jun 12, 2025
65412a9
add storybook, fix some components
koonpeng Jun 13, 2025
abac9f7
add transform edit form
koonpeng Jun 13, 2025
dc70577
enable transform form
koonpeng Jun 16, 2025
b6ce4e1
fix formatting
koonpeng Jun 17, 2025
831b8be
add registry schema
koonpeng Jun 17, 2025
e087b93
use string schema instead of creating raw schema
koonpeng Jun 17, 2025
cbc7010
use true instead of null to avoid conflict with nullable values
koonpeng Jun 17, 2025
2b424ec
use any object for forward compatibility
koonpeng Jun 17, 2025
e47c727
cleanup testing code
koonpeng Jun 18, 2025
e79bbef
Merge branch 'koonpeng/registry-schema' into koonpeng/diagram-editor
koonpeng Jun 18, 2025
752817c
update schema
koonpeng Jun 18, 2025
46deef4
update schema
koonpeng Jun 18, 2025
ec39923
Merge branch 'koonpeng/registry-schema' into koonpeng/diagram-editor
koonpeng Jun 18, 2025
16d6230
implement connections for scope and stream_out
koonpeng Jun 18, 2025
c8f93fa
fix
koonpeng Jun 18, 2025
6a27657
implement stream out for scope
koonpeng Jun 18, 2025
44a67ae
implement node for each operation independently
koonpeng Jun 18, 2025
30c5894
refactor to avoid importing from outside a package
koonpeng Jun 19, 2025
57884d0
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jun 19, 2025
631ef2e
use default export
koonpeng Jun 19, 2025
7159c9a
use correct schema version on ajv
koonpeng Jun 19, 2025
ebe4d24
use uuid as id for new nodes
koonpeng Jun 19, 2025
4b2c96a
implement delete button on all edit node forms
koonpeng Jun 19, 2025
107296c
relax package private imports to allow sibliing modules
koonpeng Jun 19, 2025
cb63e9b
add transform form story
koonpeng Jun 19, 2025
495d3df
implement edge forms
koonpeng Jun 19, 2025
5c46541
handle exporting stream out edges for scope
koonpeng Jun 19, 2025
6219864
handle deleting edges
koonpeng Jun 19, 2025
c5dfdff
fix form not updating
koonpeng Jun 19, 2025
fc8d3d2
cleanup
koonpeng Jun 19, 2025
5fc2689
more preprocessing for json-schema-to-typescript
koonpeng Jun 20, 2025
90d90d7
manage node id and op id separately to avoid ui glitches and unnecesa…
koonpeng Jun 24, 2025
78fa3ef
update deps
koonpeng Jun 25, 2025
2851a73
exclude build script from the crate
koonpeng Jun 26, 2025
447e973
use lookup maps
koonpeng Jun 26, 2025
76b6ab0
validate that the cel is valid
koonpeng Jun 26, 2025
b5aba98
fix auto layout
koonpeng Jun 26, 2025
0b5b00f
perf optimization of the command panel
koonpeng Jun 26, 2025
65d0f09
upgrade to material symbols
koonpeng Jun 30, 2025
df01814
fix node height to avoid recomputing dimensions causing lag spikes
koonpeng Jun 30, 2025
763bb3f
fix lint
koonpeng Jun 30, 2025
56f4a5b
fix and improve auto layout algorithm
koonpeng Jun 30, 2025
9126b0e
fix node width
koonpeng Jul 1, 2025
086aedb
ignore node_modules
koonpeng Jul 1, 2025
a8ed2bc
memoized edges and nodes
koonpeng Jul 1, 2025
81d0e48
revert testing changes
koonpeng Jul 1, 2025
a7ac79f
add background
koonpeng Jul 1, 2025
c3655eb
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jul 2, 2025
5a28055
serve registry
koonpeng Jul 2, 2025
f2e02e1
serve and provide registry
koonpeng Jul 3, 2025
60388c7
add script to start backend
koonpeng Jul 3, 2025
8351bd5
fix
koonpeng Jul 3, 2025
f0e122e
assume regsitry is always available
koonpeng Jul 3, 2025
a205c7e
autocomplete node builder
koonpeng Jul 4, 2025
7978c24
fix incremental build
koonpeng Jul 4, 2025
0f03cce
wip server executor
koonpeng Jul 4, 2025
15b95f6
first executor service
koonpeng Jul 7, 2025
db61bab
use existing app
koonpeng Jul 7, 2025
c7cc4bf
add placeholder debug api
koonpeng Jul 8, 2025
ed631dc
add bin to print schemas
koonpeng Jul 8, 2025
ca58219
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jul 9, 2025
955a660
use schema from the api
koonpeng Jul 9, 2025
2c2f958
able to render scopes but autoLayout not working
koonpeng Jul 11, 2025
8f0d5ce
fix tests
koonpeng Jul 11, 2025
40e824e
workable scope layout
koonpeng Jul 11, 2025
fa30535
autolayout now auto calculate scope size
koonpeng Jul 14, 2025
d51c93e
finally functional scope sizing
koonpeng Jul 15, 2025
bb1ee1e
fixes
koonpeng Jul 16, 2025
1057be1
support add node into scope
koonpeng Jul 16, 2025
ef86e52
add namespace for new operations
koonpeng Jul 16, 2025
d406a97
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jul 16, 2025
bb65138
increase handle size and selection area
koonpeng Jul 17, 2025
51e098e
support exporting diagram with scope
koonpeng Jul 17, 2025
f290d4b
reduce impact of resizing glitches
koonpeng Jul 17, 2025
d1c98a6
remove children when parent is removed
koonpeng Jul 17, 2025
e4f7ef6
support adding scope operation
koonpeng Jul 17, 2025
c9cea9a
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Jul 21, 2025
23f7ee9
remove old schema
koonpeng Jul 21, 2025
4aa106d
wip templates dialog
koonpeng Jul 23, 2025
80eebe8
implemented adding and deleting templates
koonpeng Jul 24, 2025
df5cccd
wip adding template mode for diagram editor
koonpeng Jul 24, 2025
f445cd7
increase edge contrast
koonpeng Jul 25, 2025
dd8ce56
refactor
koonpeng Jul 25, 2025
d08543a
add section interface nodes to add operation menu
koonpeng Jul 25, 2025
9b35b49
fixes to exporting in preparation for exporting templates
koonpeng Jul 28, 2025
0a624fb
more fixes for templates
koonpeng Jul 28, 2025
fe28715
saving and reloading nodes and edges when switch between template and…
koonpeng Jul 28, 2025
eb5fa9d
stricter type checks for section interfaces
koonpeng Jul 29, 2025
430a61a
ellipsis long labels, increase node width, add comment on why it is h…
koonpeng Jul 29, 2025
c7902fa
wip exporting template
koonpeng Jul 29, 2025
95b2969
rename to avoid name ambiguity
koonpeng Jul 30, 2025
403bd70
successfully export template
koonpeng Jul 30, 2025
7f730d1
successfully loaded a template
koonpeng Jul 30, 2025
710d12e
save template on close
koonpeng Jul 30, 2025
b5ef3a7
check if edge is valid when connecting; add arrow marker to edges
koonpeng Jul 31, 2025
f59b48c
more checks when connecting edges
koonpeng Jul 31, 2025
b3b2acc
do not flag conflict if the edge to check is already in reactflow
koonpeng Jul 31, 2025
8d1fa77
fix several edge validation errors
koonpeng Aug 1, 2025
767277d
formatting
koonpeng Aug 1, 2025
1549978
visual indication if an edge is connectable
koonpeng Aug 1, 2025
d23e22e
fix more validation bugs
koonpeng Aug 4, 2025
d2e5294
add forms to edit section interfaces
koonpeng Aug 4, 2025
5ab12f1
form to edit a section
koonpeng Aug 4, 2025
f5178d7
wip supporting section connections
koonpeng Aug 4, 2025
fc47a7f
implemented forms for connecting section buffers
koonpeng Aug 5, 2025
5fdc12f
added support for editing section edges
koonpeng Aug 6, 2025
d31a114
add more connection tests
koonpeng Aug 7, 2025
bc73def
refactor
koonpeng Aug 7, 2025
2ef857e
cleanup
koonpeng Aug 7, 2025
a6a96b5
color code buffer handles; join node no longer allows data edges
koonpeng Aug 8, 2025
5f548bc
add caption to section interface nodes
koonpeng Aug 8, 2025
9b1ea9e
add script to check typescript
koonpeng Aug 8, 2025
b038c37
color code nodes with multiple output and input types
koonpeng Aug 8, 2025
98dcaa5
add color coding to other nodes
koonpeng Aug 8, 2025
031c86c
cleanup
koonpeng Aug 8, 2025
4543a32
add README
koonpeng Aug 8, 2025
10aaf46
test png
koonpeng Aug 8, 2025
d66435a
try webp
koonpeng Aug 8, 2025
6c9942a
remoev test
koonpeng Aug 8, 2025
9397cf3
setup pnpm
koonpeng Aug 12, 2025
b6bfba8
wip trace backend
koonpeng Aug 13, 2025
e4c869b
run workflow ui
koonpeng Aug 14, 2025
289d80c
add config field to node form
koonpeng Aug 14, 2025
c396b6d
empty request by default
koonpeng Aug 15, 2025
1694585
ignore wip test
koonpeng Aug 15, 2025
9d646d5
styling
koonpeng Aug 15, 2025
0552be9
Make calculator operations more flexible
mxgrey Aug 17, 2025
a87cbb1
Make config error more clear
mxgrey Aug 17, 2025
ef832c9
fix formatting
mxgrey Aug 17, 2025
eb17866
fix formatting
mxgrey Aug 17, 2025
4807ac6
fix type generation
koonpeng Aug 18, 2025
1ebf670
fix ui bugs
koonpeng Aug 18, 2025
1a37bee
fix buffer input slot not updating
koonpeng Aug 18, 2025
bbaecb7
Merge branch 'koonpeng/diagram-editor' of github.com:open-rmf/bevy_im…
koonpeng Aug 18, 2025
dabc711
update icons
koonpeng Aug 18, 2025
cf14bf0
Merge remote-tracking branch 'origin/main' into koonpeng/diagram-editor
koonpeng Aug 18, 2025
71f8bde
fix after merge
koonpeng Aug 18, 2025
1a206b8
fix uncaught exceptions
koonpeng Aug 18, 2025
c422eaf
add instructions to install pnpm
koonpeng Aug 18, 2025
ad47632
proper stream out forms
koonpeng Aug 18, 2025
3de5243
separate stream out to another handle
koonpeng Aug 18, 2025
4382ec2
fixes to sections and improve default values
koonpeng Aug 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/ci_linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ jobs:
- name: Setup rust
run: rustup default ${{ matrix.rust-version }}

- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: latest

- name: Setup nodejs
run: pnpm env use --global lts

- name: Build default features
run: cargo build --workspace
- name: Test default features
Expand Down
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,14 @@ debug/
target/

Cargo.lock

# nodejs
node_modules

# diagram-editor
/diagram-editor/dist
/diagram-editor/dist.tar.gz

# storybook
*storybook.log
storybook-static
2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,10 @@ test-log = { version = "0.2.16", features = [

[workspace]
members = [

"examples/diagram/calculator",
"examples/zenoh-examples",
"diagram-editor",
]

[[bin]]
Expand Down
63 changes: 63 additions & 0 deletions biome.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
{
"$schema": "https://biomejs.dev/schemas/2.0.5/schema.json",
"files": {
"includes": [
"**/*.tsx",
"**/*.ts",
"**/*.mjs",
"**/*.cjs",
"**/*.json",
"**/*.css",
"!**/*.schema.json",
"!node_modules/**"
]
},
"assist": { "actions": { "source": { "organizeImports": "on" } } },
"vcs": {
"enabled": true,
"clientKind": "git",
"useIgnoreFile": true
},
"formatter": {
"indentStyle": "space"
},
"javascript": {
"formatter": {
"quoteStyle": "single"
}
},
"css": {
"parser": {
"cssModules": true
}
},
"linter": {
"enabled": true,
"rules": {
"recommended": true,
"correctness": {
"useExhaustiveDependencies": {
"level": "error",
"options": {
"hooks": [
{ "name": "useEditorMode", "stableResult": [1] },
{ "name": "useTemplates", "stableResult": [1] }
]
}
}
}
}
},
"overrides": [
{
"includes": ["**/*.test.ts", "**/*.test.tsx"],
"linter": {
"rules": {
"style": {
"noNonNullAssertion": "off"
}
}
}
}
]
}
4 changes: 4 additions & 0 deletions diagram-editor/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/dist

*storybook.log
storybook-static
11 changes: 11 additions & 0 deletions diagram-editor/.storybook/main.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import type { StorybookConfig } from 'storybook-react-rsbuild';

const config: StorybookConfig = {
stories: [
'../frontend/**/*.mdx',
'../frontend/**/*.stories.@(js|jsx|mjs|ts|tsx)',
],
addons: [],
framework: 'storybook-react-rsbuild',
};
export default config;
14 changes: 14 additions & 0 deletions diagram-editor/.storybook/preview.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import type { Preview } from 'storybook-react-rsbuild';

const preview: Preview = {
parameters: {
controls: {
matchers: {
color: /(background|color)$/i,
date: /Date$/i,
},
},
},
};

export default preview;
67 changes: 67 additions & 0 deletions diagram-editor/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
[package]
name = "bevy_impulse_diagram_editor"
version = "0.0.1"
edition = "2021"
authors = ["Teo Koon Peng <[email protected]>"]
license = "Apache-2.0"
description = "Frontend for bevy_impulse diagrams"
readme = "README.md"
repository = "https://github.com/open-rmf/bevy_impulse"
keywords = [
"reactive",
"workflow",
"behavior",
"agent",
"bevy",
"frontend",
"diagram",
]
categories = [
"science::robotics",
"asynchronous",
"concurrency",
"game-development",
]
exclude = ["/build.rs"]

[lib]
path = "server/lib.rs"

[dependencies]
axum = { version = "0.8.4", features = ["ws"] }
bevy_app = "0.12.1"
bevy_ecs = "0.12.1"
bevy_impulse = { version = "0.0.2", path = "..", features = [
"diagram",
"trace",
] }
flate2 = { version = "1.1.1", optional = true }
futures-util = "0.3.31"
indexmap = { version = "2.10.0", optional = true, features = ["serde"] }
mime_guess = "2.0.5"
schemars = { version = "0.9", optional = true }
serde = "1.0.219"
serde_json = "1.0.140"
tar = { version = "0.4.44", optional = true }
tokio = "1.45.1"
tracing = "0.1.41"

[build-dependencies]
flate2 = { version = "1.1.1", optional = true }
tar = { version = "0.4.44", optional = true }

[dev-dependencies]
futures-channel = "0.3.31"
test-log = "0.2.18"
tokio = { version = "1.45.1", features = ["macros", "rt-multi-thread"] }
tower = "0.5.2"

[features]
default = ["frontend"]
frontend = ["dep:flate2", "dep:tar"]
json_schema = ["dep:schemars", "dep:indexmap"]

[[bin]]
name = "print_schema"
path = "server/bin/print_schema.rs"
required-features = ["json_schema"]
77 changes: 77 additions & 0 deletions diagram-editor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# bevy_impulse_diagram_editor

![](./docs/assets/diagram-editor-preview.webp)

This contains a SPA React web app to create and edit a `bevy_impulse` diagram and an axum router to serve it.

## Setup

Install pnpm and nodejs:

```bash
curl -fsSL https://get.pnpm.io/install.sh | bash -
pnpm env use --global lts
```

Install the dependencies:

```bash
pnpm install
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's include the instructions for setting up pnpm like what we have for rmf-web. Our target user base is the ROS community, who often have no experience using web tools like pnpm.

```

## Embedding the Diagram Editor into a `bevy_impulse` app

The frontend is built using `rsbuild` and embedded inside the crate. The library exposes an axum router that can be used to serve both the frontend and backend:

```rs
use bevy_impulse_diagram_editor::{new_router, ServerOptions};

fn main() {
let mut registry = DiagramElementRegistry::new();
// register node builders, section builders etc.

let mut app = bevy_app::App::new();
app.add_plugins(ImpulseAppPlugin::default());
let router = new_router(&mut app, registry, ServerOptions::default());
let listener = tokio::net::TcpListener::bind(("localhost", 3000))
.await
.unwrap();
axum::serve(listener, router).await?;
}
```

To omit the frontend and serve only the backend API, disable the default features:

```toml
[dependencies]
bevy_impulse_diagram_editor = { version = "0.0.1", default-features = false }
```

See the [calculator demo](../examples/diagram/calculator) for more examples.

## Local development server

Normally the web stack is not required by using this crate as a dependency, but it is required when developing the frontend.

Requirements:

* nodejs
* pnpm

First start the `dev` backend server:

```bash
pnpm dev:backend
```

then in another terminal, start the frontend `dev` server:

```bash
pnpm dev
```

When there are breaking changes in `bevy_impulse`, the typescript definitions need to be regenerated:

```bash
pnpm generate-types
```
75 changes: 75 additions & 0 deletions diagram-editor/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/// Builds the frontend and packages it into a tar.gz archive.
/// This requires `pnpm` and all the js dependencies to be available.
///
/// This build script is excluded from the crate, the output tarball is included instead.
/// This allows downstream to build the crate without any of the js stack.

#[cfg(feature = "frontend")]
mod frontend {
use flate2::{write::GzEncoder, Compression};
use std::fs::File;
use std::path::PathBuf;
use std::process::Command;
use tar::Builder;

pub fn build_frontend() {
println!("cargo:rerun-if-changed=build.rs");
println!("cargo:rerun-if-changed=package.json");
println!("cargo:rerun-if-changed=../pnpm-lock.yaml");
println!("cargo:rerun-if-changed=rsbuild.config.ts");
println!("cargo:rerun-if-changed=frontend");

let status = Command::new("pnpm")
.arg("install")
.arg("--frozen-lockfile")
.status()
.expect("Failed to execute pnpm install");

if !status.success() {
panic!("pnpm install failed with status: {:?}", status);
}

let status = Command::new("pnpm")
.arg("build")
.status()
.expect("Failed to execute pnpm build");

if !status.success() {
panic!("pnpm build failed with status: {:?}", status);
}

let dist_dir_path = "dist";
// We put the output in `CARGO_MANIFEST_DIR` instead of `OUT_DIR` because we want to include
// it in the crate.
let out_dir =
PathBuf::from(std::env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR not set"));
let output_tar_gz_path = out_dir.join("dist.tar.gz");

if std::path::Path::new(dist_dir_path).exists() {
let tar_gz_file = File::create(&output_tar_gz_path)
.expect("Failed to create output tar.gz file in CARGO_MANIFEST_DIR");
let enc = GzEncoder::new(tar_gz_file, Compression::default());
let mut tar_builder = Builder::new(enc);

// Add the entire "dist" directory to the archive, preserving its name within the archive.
tar_builder
.append_dir_all(".", dist_dir_path)
.expect("Failed to add directory to tar archive");
tar_builder.finish().expect("Failed to finish tar archive");
println!(
"Successfully compressed '{}' into '{:?}'",
dist_dir_path, output_tar_gz_path
);
} else {
panic!(
"Directory '{}' not found after pnpm build. Frontend build might have failed.",
dist_dir_path
);
}
}
}

fn main() {
#[cfg(feature = "frontend")]
frontend::build_frontend();
}
Binary file not shown.
Loading
Loading