Skip to content

feat(integration-tests): Add end-to-end test workflow for downstream clp-ffi-js; Bump yscope-dev-utils to y-scope/yscope-dev-utils@e2a1aed. #2015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Bill-hbrhbr wants to merge 14 commits into
base: main
Choose a base branch
from
Open

feat(integration-tests): Add end-to-end test workflow for downstream clp-ffi-js; Bump yscope-dev-utils to y-scope/yscope-dev-utils@e2a1aed. #2015

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
7d70548
Add clp-ffi-js test
Bill-hbrhbr Feb 23, 2026
b06105c
Bump yscope-dev-utils
Bill-hbrhbr Feb 23, 2026
3b28ce6
Polish with docstring
Bill-hbrhbr Feb 23, 2026
94e3a2b
Refactor
Bill-hbrhbr Feb 24, 2026
8a8feb2
lint fix
Bill-hbrhbr Feb 24, 2026
f336c64
Merge branch 'main' into integration-tests/clp-ffi-js
Bill-hbrhbr Feb 24, 2026
eedf82e
polish
Bill-hbrhbr Feb 24, 2026
999f11a
Merge branch 'main' into integration-tests/clp-ffi-js
Bill-hbrhbr Feb 25, 2026
90e9647
Address coderabbit review
Bill-hbrhbr Feb 25, 2026
a47a3f0
Merge branch 'main' into integration-tests/clp-ffi-js
Bill-hbrhbr Feb 25, 2026
8eb916f
Merge branch 'main' into integration-tests/clp-ffi-js
Bill-hbrhbr Feb 25, 2026
8956e06
Merge branch 'main' into integration-tests/clp-ffi-js
Bill-hbrhbr Feb 26, 2026
9bfb82e
Merge branch 'main' into integration-tests/clp-ffi-js
hoophalab Mar 2, 2026
ab0507d
Merge branch 'main' into integration-tests/clp-ffi-js
junhaoliao Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions taskfiles/tests/clp-ffi-js.yaml
View file
Open in desktop
Copy link
Member

@junhaoliao junhaoliao Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there's no easy way to set up clp-ffi-js as one of deps for build-and-test-node and build-and-test-browser, how do you feel about adding this to both tasks:

    preconditions:
      - msg: "clp-ffi-js is not set up. Run `task tests:clp-ffi-js` first."
        sh: "test -d {{.G_CLP_FFI_JS_BUILD_DIR}}"

Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
version: "3"

includes:
yscope-dev-utils: "../../tools/yscope-dev-utils/exports/taskfiles/utils/utils.yaml"

vars:
G_CLP_FFI_JS_SRC_DIR: "{{.G_BUILD_DIR}}/clp-ffi-js"
G_CLP_FFI_JS_BUILD_DIR: "{{.G_CLP_FFI_JS_SRC_DIR}}/build/clp-ffi-js"
G_CLP_FFI_JS_NODE_MODULE_PATH: "{{.G_CLP_FFI_JS_BUILD_DIR}}/ClpFfiJs-node.js"
G_CLP_FFI_JS_WORKER_MODULE_PATH: "{{.G_CLP_FFI_JS_BUILD_DIR}}/ClpFfiJs-worker.js"

G_CLP_FFI_JS_CHECKSUM_FILE: "{{.G_BUILD_DIR}}/clp-ffi-js.md5"

# NOTE: clp-ffi-js normally builds its WASM artifacts against a pinned CLP source revision. For
# local CLP development and integration testing, we replace that dependency with the current
# workspace CLP checkout via a symlink.
#
# This allows us to quickly verify whether in-flight CLP changes remain compatible with downstream
# clp-ffi-js while reusing the existing clp-ffi-js build and test workflow through a small set of
# hacks/overrides.
#
# Run `task:clp-ffi-js` once to perform the initial setup. After that, selectively run
# - `task:clp-ffi-js:build-and-test-node`
# - `task:clp-ffi-js:build-and-test-browser`
# as needed. Note that `sudo` access is required for the latter.
tasks:
default:
dir: "{{.G_CLP_FFI_JS_SRC_DIR}}"
deps: ["download-src"]
cmds:
# Expose `clp-ffi-js` tasks for test setup
- task: "symlink-yscope-dev-utils"
Copy link
Contributor

@hoophalab hoophalab Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we make symlink-yscope-dev-utils a separate sub-task while keep

        rm -rf build/deps/clp
        ln -s "{{.ROOT_DIR}}" build/deps/clp

inline? Might be cleaner to make both inline.

- |-
sed -i 's/^\(\s*internal:\s*\)true\b/\1false/' taskfile.yaml
Copy link
Member

@junhaoliao junhaoliao Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given the tasks in clp-ffi-js is now publicly (external to the clp-ffi-js) used, maybe we should change the tasks in clp-ffi-js to non-internal?

so long we dont document those tasks in clp-ffi-js docs, it shouldn't cause confusion to developers about what tasks they should run? if you really want, i think we can add inline docstrings to those tasks to explain thse are needed to set up the right dependencies for integration tests in the clp project. either way, it should be cleaner / more future-proof than this sed here

task node-modules
task config-cmake-project
rm -rf build/deps/clp
ln -s {{.ROOT_DIR}} build/deps/clp

# Repeatable test workflow
- task: "build-and-test-node"
- task: "build-and-test-browser"

build-and-test-node:
dir: "{{.G_CLP_FFI_JS_SRC_DIR}}"
deps: ["build-targets"]
cmds:
- |-
VITE_NODE_MODULE_ABS_PATH="{{.G_CLP_FFI_JS_NODE_MODULE_PATH}}" \
npm run test -- --project node

# NOTE: Browser tests require installing browsers and system dependencies. The task is skipped if
# sudo is unavailable.
Copy link
Member

@junhaoliao junhaoliao Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# sudo is unavailable.
# passwordless sudo is unavailable.

build-and-test-browser:
dir: "{{.G_CLP_FFI_JS_SRC_DIR}}"
deps: ["build-targets"]
status:
- "! (command -v sudo >/dev/null && sudo -n true)"
cmds:
- |-
Comment on lines +57 to +60
Copy link
Member

@junhaoliao junhaoliao Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
status:
- "! (command -v sudo >/dev/null && sudo -n true)"
cmds:
- |-
cmds:
- |-
if ! (command -v sudo >/dev/null && sudo -n true 2>/dev/null); then
echo "WARNING: Skipping browser tests (passwordless sudo is unavailable)."
exit 0
fi

npm run test:init
VITE_WORKER_MODULE_ABS_PATH="{{.G_CLP_FFI_JS_WORKER_MODULE_PATH}}" \
npm run test -- --project browser

# Builds clp-ffi-js + CLP with Emscripten + Clang
build-targets:
internal: true
run: "once"
cmds:
- |-
cmake \
--build "{{.G_CLP_FFI_JS_BUILD_DIR}}" \
--parallel \
--target "ClpFfiJs-node" "ClpFfiJs-worker"

download-src:
internal: true
deps:
- "::init"
cmds:
- task: "yscope-dev-utils:remote:download-and-extract-tar"
vars:
CHECKSUM_FILE: "{{.G_CLP_FFI_JS_CHECKSUM_FILE}}"
FILE_SHA256: "5ab1c27031caafb014198d4db711a71b3fbb1661b396bcd4e46764194e2b1f61"
OUTPUT_DIR: "{{.G_BUILD_DIR}}/clp-ffi-js"
TAR_FILE: "{{.G_BUILD_DIR}}/clp-ffi-js.tar.gz"
URL: "https://github.com/y-scope/clp-ffi-js/archive/4cc6a7c.tar.gz"
Copy link
Member

@junhaoliao junhaoliao Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think it's worth adding a docstring to explain when this commit was available? / in the future if we pin those commit ids at specific clp-ffi-js releases, it's better to comment with the release versions as well

Copy link
Member

@junhaoliao junhaoliao Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something like

# clp-ffi-js @ v0.X.Y (March 2, 2026)


symlink-yscope-dev-utils:
internal: true
cmds:
- |-
rm -rf "{{.G_CLP_FFI_JS_SRC_DIR}}/tools/yscope-dev-utils"
ln -s "{{.ROOT_DIR}}/tools/yscope-dev-utils" \
"{{.G_CLP_FFI_JS_SRC_DIR}}/tools/yscope-dev-utils"
1 change: 1 addition & 0 deletions taskfiles/tests/main.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
version: "3"

includes:
clp-ffi-js: "clp-ffi-js.yaml"
integration: "integration.yaml"
toolchains: "../toolchains.yaml"

Expand Down
2 changes: 1 addition & 1 deletion tools/scripts/deps-download/init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
project_root_dir="$script_dir/../../../"
download_dep_script="$script_dir/download-dep.py"

readonly YSCOPE_DEV_UTILS_COMMIT_SHA="6ec46856cfda8cbfb65d9386affbe3bc429186cb"
readonly YSCOPE_DEV_UTILS_COMMIT_SHA="e2a1aed32095b05d45b74c5cf52a9c9e176f1c40"
python3 "${download_dep_script}" \
"https://github.com/y-scope/yscope-dev-utils/archive/${YSCOPE_DEV_UTILS_COMMIT_SHA}.zip" \
"yscope-dev-utils-${YSCOPE_DEV_UTILS_COMMIT_SHA}" \
Expand Down
2 changes: 1 addition & 1 deletion tools/yscope-dev-utils
Open in desktop
Submodule yscope-dev-utils updated 1 files
+21 −23 exports/taskfiles/utils/boost.yaml
Loading