Skip to content

Commit 1704e18

Browse files
Test Job (with NodeJS) for Windows in the CI Workflow (#336)
* Write a starter environment script in PowerShell Signed-off-by: Param Siddharth <[email protected]> * Add test for Windows; Temporarily disable Linux tests Signed-off-by: Param Siddharth <[email protected]> * Update the variables Signed-off-by: Param Siddharth <[email protected]> * Mark variables as global Signed-off-by: Param Siddharth <[email protected]> * Make all variables global Signed-off-by: Param Siddharth <[email protected]> * Fix the usage of args Signed-off-by: Param Siddharth <[email protected]> * Replace apt with choco Signed-off-by: Param Siddharth <[email protected]> * Add code for Python Signed-off-by: Param Siddharth <[email protected]> * Debug environment Signed-off-by: Param Siddharth <[email protected]> * Assign to global variables Signed-off-by: Param Siddharth <[email protected]> * Path issue fix Signed-off-by: Param Siddharth <[email protected]> * Use original Python binary Signed-off-by: Param Siddharth <[email protected]> * Update script to use existing Windows CI code Signed-off-by: Param Siddharth <[email protected]> * Clarify when replacing Python Signed-off-by: Param Siddharth <[email protected]> * Check for exit code correctly (pwsh) Signed-off-by: Param Siddharth <[email protected]> * Save path differently Signed-off-by: Param Siddharth <[email protected]> * Modify PATH variable using GITHUB_PATH Signed-off-by: Param Siddharth <[email protected]> * Add runtime path to the beginning of PATH Signed-off-by: Param Siddharth <[email protected]> * Use variable for new PATH Signed-off-by: Param Siddharth <[email protected]> * Write configuration script Signed-off-by: Param Siddharth <[email protected]> * Update workflow with configuration script for Windows Signed-off-by: Param Siddharth <[email protected]> * Update function syntax for `sub-configure` Signed-off-by: Param Siddharth <[email protected]> * Fix variable name in workflow Signed-off-by: Param Siddharth <[email protected]> * Fix multiline breaking of commands, Go to build directory Signed-off-by: Param Siddharth <[email protected]> * Concatenate strings correctly Signed-off-by: Param Siddharth <[email protected]> * Create and navigate to build dir from the workflow itself Signed-off-by: Param Siddharth <[email protected]> * Uncomment `FindPython` patch Signed-off-by: Param Siddharth <[email protected]> * Show `FindPython.cmake` path and contents Signed-off-by: Param Siddharth <[email protected]> * Display errorlevel at the end Signed-off-by: Param Siddharth <[email protected]> * Upload FindPython.cmake for debugging Signed-off-by: Param Siddharth <[email protected]> * Remove accidental uncomment Signed-off-by: Param Siddharth <[email protected]> * Fix command error Signed-off-by: Param Siddharth <[email protected]> * Use `curl.exe` Signed-off-by: Param Siddharth <[email protected]> * Set default encoding Signed-off-by: Param Siddharth <[email protected]> * Remove debug statements (for `FindPython.cmake`) Signed-off-by: Param Siddharth <[email protected]> * Return with last exit code in configuration script Signed-off-by: Param Siddharth <[email protected]> * Create separate method to add to PATH Signed-off-by: Param Siddharth <[email protected]> * Add build step Signed-off-by: Param Siddharth <[email protected]> * Remove target argument from build step Signed-off-by: Param Siddharth <[email protected]> * Collect logs for `funchook` failure Signed-off-by: Param Siddharth <[email protected]> * Remove log upload, Disable fork safe build Signed-off-by: Param Siddharth <[email protected]> * Print build string before building Signed-off-by: Param Siddharth <[email protected]> * Include debug and symbols in Python installation Signed-off-by: Param Siddharth <[email protected]> * Include debug library in Python Signed-off-by: Param Siddharth <[email protected]> * Fix multiline bug Signed-off-by: Param Siddharth <[email protected]> * Refresh env after adding to PATH Signed-off-by: Param Siddharth <[email protected]> * Write a separate script for building Signed-off-by: Param Siddharth <[email protected]> * Run tests using CTest Signed-off-by: Param Siddharth <[email protected]> * Fix improper merge Signed-off-by: Param Siddharth <[email protected]> * Add newlines wherever needed Signed-off-by: Param Siddharth <[email protected]> * Add Node environment setup Signed-off-by: Param Siddharth <[email protected]> * Update workflow to set up Node Signed-off-by: Param Siddharth <[email protected]> * Remove NASM from configuration setup and add it to environment setup Signed-off-by: Param Siddharth <[email protected]> * Make dependency directory Signed-off-by: Param Siddharth <[email protected]> * Debug: Show directories Signed-off-by: Param Siddharth <[email protected]> * Debug: Show more info Signed-off-by: Param Siddharth <[email protected]> * Debug: Call GNU utils Signed-off-by: Param Siddharth <[email protected]> * Debug: ??? Signed-off-by: Param Siddharth <[email protected]> * Debug: Show parent directory Signed-off-by: Param Siddharth <[email protected]> * Remove commented out code Signed-off-by: Param Siddharth <[email protected]> * Remove './' from web client Signed-off-by: Param Siddharth <[email protected]> * Dowmload to explicit path Signed-off-by: Param Siddharth <[email protected]> * Use explicit download paths (why?) Signed-off-by: Param Siddharth <[email protected]> * Remove debug statements Signed-off-by: Param Siddharth <[email protected]> * Remove invalid parameter Signed-off-by: Param Siddharth <[email protected]> * Keep the Node DLL in the right location Signed-off-by: Param Siddharth <[email protected]> Signed-off-by: Param Siddharth <[email protected]>
1 parent 883f92e commit 1704e18

File tree

3 files changed

+83
-46
lines changed

3 files changed

+83
-46
lines changed

.github/workflows/test.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -95,14 +95,14 @@ jobs:
9595
uses: actions/checkout@v2
9696
with:
9797
fetch-depth: 0
98-
98+
9999
- name: Set up the environment
100100
run: cmd.exe /c "powershell .\tools\metacall-environment.ps1 $Env:METACALL_INSTALL_OPTIONS"
101101
env:
102102
LTTNG_UST_REGISTER_TIMEOUT: 0
103103
NUGET_XMLDOC_MODE: skip
104104
DOTNET_CLI_TELEMETRY_OPTOUT: 'true'
105-
METACALL_INSTALL_OPTIONS: base python ruby netcore5 nodejs typescript file rpc wasm java c cobol rust rapidjson funchook swig pack # clangformat v8rep51 coverage
105+
METACALL_INSTALL_OPTIONS: base python nodejs nasm # ruby netcore5 typescript file rpc wasm java c cobol rust rapidjson funchook swig pack # clangformat v8rep51 coverage
106106

107107
- name: Run the configuration script
108108
run: |
@@ -115,7 +115,7 @@ jobs:
115115
NUGET_XMLDOC_MODE: skip
116116
DOTNET_CLI_TELEMETRY_OPTOUT: 'true'
117117
METACALL_BUILD_TYPE: debug
118-
METACALL_BUILD_OPTIONS: python # root ruby netcore5 nodejs typescript file rpc wasm java c cobol rust examples tests scripts ports dynamic install pack benchmarks # sanitizer v8 coverage
118+
METACALL_BUILD_OPTIONS: python nodejs # root ruby netcore5 typescript file rpc wasm java c cobol rust examples tests scripts ports dynamic install pack benchmarks # sanitizer v8 coverage
119119

120120
- name: Build the core
121121
working-directory: ./build
@@ -125,4 +125,4 @@ jobs:
125125
NUGET_XMLDOC_MODE: skip
126126
DOTNET_CLI_TELEMETRY_OPTOUT: 'true'
127127
METACALL_BUILD_TYPE: debug
128-
METACALL_BUILD_OPTIONS: tests # coverage install
128+
METACALL_BUILD_OPTIONS: tests # coverage install

tools/metacall-build.ps1

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ function sub-build {
4545
# Tests (coverage needs to run the tests)
4646
if ( ($BUILD_TESTS -eq 1) -or ($BUILD_COVERAGE -eq 1) ) {
4747
echo "Running the tests..."
48-
ctest "-j$((Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors)" --output-on-failure --test-output-size-failed 3221000000 -C $BUILD_TYPE
48+
ctest "-j$((Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors)" --output-on-failure -C $BUILD_TYPE
4949
}
5050

5151
# Coverage
@@ -84,4 +84,4 @@ switch($args.length) {
8484
sub-options
8585
sub-build
8686
}
87-
}
87+
}

tools/metacall-environment.ps1

Lines changed: 77 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
$PSDefaultParameterValues['*:Encoding'] = 'utf8'
2+
$Global:ProgressPreference = 'SilentlyContinue'
23

34
$Global:ROOT_DIR = "$(pwd)"
45

@@ -22,6 +23,7 @@ $Global:INSTALL_NODEJS = 0
2223
$Global:INSTALL_TYPESCRIPT = 0
2324
$Global:INSTALL_FILE = 0
2425
$Global:INSTALL_RPC = 0
26+
$Global:INSTALL_NASM = 0
2527
$Global:INSTALL_WASM = 0
2628
$Global:INSTALL_JAVA = 0
2729
$Global:INSTALL_C = 0
@@ -80,49 +82,14 @@ function sub-python {
8082

8183
$PythonVersion = '3.9.7'
8284
$RuntimeDir = "$ROOT_DIR\runtimes\python"
85+
$DepsDir = "$ROOT_DIR\dependencies"
8386

84-
<#
85-
86-
# Avoiding; The Python installation provided by Chocolatey is statically compiled
87-
88-
$PythonVersion = '3.10.6'
89-
90-
choco install python3 --version $PythonVersion -my
91-
92-
$PythonPath = "$Env:ChocolateyInstall\lib\python3.$PythonVersion\tools"
93-
$PythonBin = "$PythonPath\python-$PythonVersion-amd64.exe"
94-
95-
cmd.exe /c "mklink ""$PythonPath\python.exe"" ""$PythonBin"""
96-
cmd.exe /c "mklink ""$ChocolateyBinPath\python.exe"" ""$PythonBin"""
97-
98-
setx /M PATH "$ChocolateyBinPath;$Env:PATH"
99-
$Env:PATH = "$ChocolateyBinPath;$Env:PATH"
100-
101-
refreshenv
102-
103-
# DEBUG
104-
# where.exe python
105-
# # python.exe -c "from sysconfig import get_paths as gp; print(gp()['include'])"
106-
# cmd.exe /c """$PythonBin"" -c ""from sysconfig import get_paths as gp; print(gp()['include'])"""
107-
108-
# Patch for FindPython.cmake
109-
# $FindPython = "$ROOT_DIR\cmake\FindPython.cmake"
110-
# $EscapedLoc = $ROOT_DIR.Replace('\', '/')
111-
# $PythonRuntimeDir = "$EscapedLoc/runtimes/python"
112-
113-
# echo set(Python_VERSION $PythonVersion) > $FindPython
114-
# echo set(Python_ROOT_DIR "$PythonRuntimeDir") >> $FindPython
115-
# echo set(Python_EXECUTABLE "%$PythonRuntimeDir/python.exe") >> $FindPython
116-
# echo set(Python_INCLUDE_DIRS "%$PythonRuntimeDir/include") >> $FindPython
117-
# echo set(Python_LIBRARIES "%$PythonRuntimeDir/libs/python39.lib") >> $FindPython
118-
# echo include(FindPackageHandleStandardArgs)>> $FindPython
119-
# echo FIND_PACKAGE_HANDLE_STANDARD_ARGS(Python REQUIRED_VARS Python_EXECUTABLE Python_LIBRARIES Python_INCLUDE_DIRS VERSION_VAR Python_VERSION) >> $FindPython
120-
# echo mark_as_advanced(Python_EXECUTABLE Python_LIBRARIES Python_INCLUDE_DIRS) >> $FindPython
121-
122-
#>
87+
md -Force $DepsDir
88+
md -Force $RuntimeDir
89+
cd $DepsDir
12390

12491
# Download installer
125-
(New-Object Net.WebClient).DownloadFile("https://www.python.org/ftp/python/$PythonVersion/python-$PythonVersion-amd64.exe", './python_installer.exe')
92+
(New-Object Net.WebClient).DownloadFile("https://www.python.org/ftp/python/$PythonVersion/python-$PythonVersion-amd64.exe", "$(pwd)\python_installer.exe")
12693

12794
# Install Python
12895
where.exe /Q python
@@ -220,7 +187,54 @@ function sub-nodejs {
220187
# TODO (copied from metacall-environment.sh): Review conflicts with Ruby Rails and NodeJS 4.x
221188
echo "configure nodejs"
222189
cd $ROOT_DIR
190+
191+
$DepsDir = "$ROOT_DIR\dependencies"
192+
$NodeVersion = '14.18.2'
193+
$DLLReleaseVer = 'v0.0.1'
194+
$RuntimeDir = "$ROOT_DIR\runtimes\nodejs"
195+
196+
md -Force $DepsDir
197+
md -Force $RuntimeDir
198+
cd $DepsDir
199+
200+
# Download
201+
(New-Object Net.WebClient).DownloadFile("https://nodejs.org/download/release/v$NodeVersion/node-v$NodeVersion-win-x64.zip", "$(pwd)\node.zip")
202+
(New-Object Net.WebClient).DownloadFile("https://nodejs.org/download/release/v$NodeVersion/node-v$NodeVersion-headers.tar.gz", "$(pwd)\node_headers.tar.gz")
223203

204+
# Install runtime
205+
Expand-Archive -Path "node.zip" -DestinationPath $RuntimeDir
206+
robocopy /move /e "$RuntimeDir\node-v$NodeVersion-win-x64" "$RuntimeDir" /NFL /NDL /NJH /NJS /NC /NS /NP
207+
rd "$RuntimeDir\node-v$NodeVersion-win-x64"
208+
209+
Add-to-Path $RuntimeDir
210+
211+
# Install headers
212+
cmake -E tar xzf node_headers.tar.gz
213+
cd "$DepsDir\node-v$NodeVersion"
214+
md "$RuntimeDir\include"
215+
robocopy /move /e "$DepsDir\node-v$NodeVersion\include" "$RuntimeDir\include" /NFL /NDL /NJH /NJS /NC /NS /NP
216+
cd $DepsDir
217+
rd -Recurse -Force "$DepsDir\node-v$NodeVersion"
218+
219+
# Install custom Node DLL
220+
(New-Object Net.WebClient).DownloadFile("https://github.com/metacall/node.dll/releases/download/$DLLReleaseVer/node-shared-v$NodeVersion-x64.zip", "$(pwd)\node_dll.zip")
221+
Expand-Archive -Path "node_dll.zip" -DestinationPath "$RuntimeDir\lib"
222+
223+
# Patch for FindNodeJS.cmake
224+
$FindNode = "$ROOT_DIR\cmake\FindNodeJS.cmake"
225+
$NodeDir = $RuntimeDir.Replace('\', '/')
226+
227+
echo "set(NodeJS_VERSION $NodeVersion)" >> $FindNode
228+
echo "set(NodeJS_INCLUDE_DIRS ""$NodeDir/include/node"")" >> $FindNode
229+
echo "set(NodeJS_LIBRARY ""$NodeDir/lib/libnode.lib"")" >> $FindNode
230+
echo "set(NodeJS_EXECUTABLE ""$NodeDir/node.exe"")" >> $FindNode
231+
echo "include(FindPackageHandleStandardArgs)" >> $FindNode
232+
echo "FIND_PACKAGE_HANDLE_STANDARD_ARGS(NodeJS REQUIRED_VARS NodeJS_INCLUDE_DIRS NodeJS_LIBRARY NodeJS_EXECUTABLE VERSION_VAR NodeJS_VERSION)" >> $FindNode
233+
echo "mark_as_advanced(NodeJS_VERSION NodeJS_INCLUDE_DIRS NodeJS_LIBRARY NodeJS_EXECUTABLE)" >> $FindNode
234+
235+
# Move DLL to correct location (to be done AFTER build)
236+
# mv -Force "$RuntimeDir\lib\libnode.dll" "$ROOT_DIR\lib"
237+
cp -Force "$RuntimeDir\lib\libnode.dll" "$ROOT_DIR\lib"
224238
}
225239

226240
# TypeScript
@@ -241,6 +255,22 @@ function sub-rpc {
241255

242256
}
243257

258+
# NASM
259+
function sub-nasm {
260+
echo "configure nasm"
261+
cd $ROOT_DIR
262+
263+
$NASMVer = '2.15.05'
264+
265+
(New-Object Net.WebClient).DownloadFile("https://www.nasm.us/pub/nasm/releasebuilds/$NASMVer/win64/nasm-$NASMVer-win64.zip", "$(pwd)\nasm.zip")
266+
Expand-Archive -Path 'nasm.zip' -DestinationPath .
267+
268+
$NASMDir = "$ROOT_DIR\nasm-$NASMVer"
269+
270+
Add-to-Path "$NASMDir\rdoff"
271+
Add-to-Path $NASMDir
272+
}
273+
244274
# WebAssembly
245275
function sub-wasm {
246276
echo "configure webassembly"
@@ -329,6 +359,9 @@ function sub-install {
329359
if ( $INSTALL_V8REPO -eq 1 ) {
330360
sub-v8repo
331361
}
362+
if ( $INSTALL_NASM -eq 1 ) {
363+
sub-nasm
364+
}
332365
if ( $INSTALL_NODEJS -eq 1 ) {
333366
sub-nodejs
334367
}
@@ -456,6 +489,10 @@ function sub-options {
456489
echo "wasm selected"
457490
$Global:INSTALL_WASM = 1
458491
}
492+
if ( "$var" -eq 'nasm' ) {
493+
echo "nasm selected"
494+
$Global:INSTALL_NASM = 1
495+
}
459496
if ( "$var" -eq 'java' ) {
460497
echo "java selected"
461498
$Global:INSTALL_JAVA = 1

0 commit comments

Comments
 (0)