Error compiling pistache on windows

[Kreijstal]

Hello, it's me again, I am trying in another windows machine, (Without admin rights). I can install msys2 tools, so I tried reproducing the same steps, but maybe there is something going around with powershell, this was however on win10 and not in win11

"C:/rtools44/ucrt64/bin/meson" "--internal" "exe" "--unpickle" "C:/rtools44/home/SYSTEM/pistache/bui
ld/meson-private/meson_exe_powershell.exe_603cd33c7e8fa4e3e9a54f8f255f85658890ce9b.dat"
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:/rtools44/ucrt64/bin/meson.exe/__main__.py", line 7, in <module>
  File "C:/rtools44/ucrt64/lib/python3.12/site-packages/mesonbuild/mesonmain.py", line 293, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/rtools44/ucrt64/lib/python3.12/site-packages/mesonbuild/mesonmain.py", line 281, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/rtools44/ucrt64/lib/python3.12/site-packages/mesonbuild/mesonmain.py", line 222, in run_s
cript_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/rtools44/ucrt64/lib/python3.12/site-packages/mesonbuild/scripts/meson_exe.py", line 112,
in run
    return run_exe(exe)
           ^^^^^^^^^^^^
  File "C:/rtools44/ucrt64/lib/python3.12/site-packages/mesonbuild/scripts/meson_exe.py", line 78, i
n run_exe
    print(stderr.decode(encoding=encoding, errors='replace'))
  File "C:/rtools44/ucrt64/lib/python3.12/encodings/cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position 176: character maps
to <undefined>
while executing ['C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe', '-NoProfile', '-N
onInteractive', '-WindowStyle', 'Hidden', '-NoLogo', '-Command', 'C:/rtools44/home/SYSTEM/pistache/s
rc/winlog/installman.ps1', '-dirinpstlogdll', 'C:/rtools44/home/SYSTEM/pistache/build', '-inpstlogdl
l', 'src/libpistachelog.dll', '-inpstman', '../src/winlog/pist_winlog.man', '-outpstmaninst', 'src/p
ist_winlog_inst_record.txt']
--- stdout ---

[Kreijstal]

It still compiles if I simply take the file out

diff --git a/src/meson.build b/src/meson.build
index 827e523..14f4744 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -304,7 +304,7 @@ if host_machine.system() == 'windows'

 endif

-if host_machine.system() == 'windows'
+if host_machine.system() == 'windows' and compiler.get_id() == 'msvc'
     # Next we install the logging manifest and pistachelog.dll in the
     # system. We do this (install logging components alone, without
     # Pistache as a whole) as part of meson build, not just as part of

it seems to fail 2 tests, however

INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\rtools44\ucrt64\bin/ninja.EXE -C C:/rtools44/home/SYSTE
M/pistache/build
ninja: Entering directory `C:/rtools44/home/SYSTEM/pistache/build'
[1/116] Generating src/gen-mc-file with a custom command
Successfully wrote to src/pist_winlog.mc
[116/116] Linking target examples/runrest_description.exe

tub+kreijstal@F6-CIP29-A-48 UCRT64 /home/SYSTEM/pistache
$ meson test -C build
ninja: Entering directory `C:/rtools44/home/SYSTEM/pistache/build'
ninja: no work to do.
1/28 pistache / async_test                       OK              12.15s
2/28 pistache / cookie_test                      OK               0.23s
3/28 pistache / cookie_test_2                    OK               0.22s
4/28 pistache / cookie_test_3                    OK               0.27s
5/28 pistache / headers_test                     OK               0.26s
6/28 pistache / http_client_test                 OK               8.37s
7/28 pistache / http_parsing_test                OK               0.23s
8/28 pistache / http_server_test                 OK              28.70s
9/28 pistache / http_uri_test                    OK               0.58s
10/28 pistache / listener_test                    OK               0.26s
11/28 pistache / log_api_test                     OK               0.23s
12/28 pistache / mailbox_test                     OK               0.22s
13/28 pistache / mime_test                        OK               0.24s
14/28 pistache:network / net_test                 OK               0.25s
15/28 pistache / reactor_test                     OK               1.25s
16/28 pistache / request_size_test                OK               0.63s
17/28 pistache / rest_server_test                 FAIL            22.95s   exit status 1
>>> MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 PATH=C:/rtools4
4/home/SYSTEM/pistache/build/src;C:/rtools44/ucrt64/bin;C:\rtools44\ucrt64\bin;C:\rtools44\usr\local
\bin;C:\rtools44\usr\bin;C:\rtools44\usr\bin;C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\progra~1\git\cmd;C:\rtools44\usr\bin\site_perl;C:\rt
ools44\usr\bin\vendor_perl;C:\rtools44\usr\bin\core_perl;\\tubfs.tu-berlin.de\kreijstal\.fzf\bin UBS
AN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 ASAN_OPTIONS=halt_on_
error=1:abort_on_error=1:print_summary=1 MESON_TEST_ITERATION=1 MALLOC_PERTURB_=82 C:/rtools44/home/
SYSTEM/pistache/build/tests/run_rest_server_test.exe

18/28 pistache / rest_swagger_server_test         OK               4.39s
19/28 pistache / router_test                      OK              20.90s
20/28 pistache / stream_test                      OK               0.44s
21/28 pistache / streaming_test                   OK               7.03s
22/28 pistache / string_logger_test               OK               0.23s
23/28 pistache / threadname_test                  OK               0.28s
24/28 pistache / typeid_test                      OK               0.22s
25/28 pistache / view_test                        OK               0.22s
26/28 pistache / helpers_test                     OK               0.23s
27/28 pistache / https_server_test                FAIL             0.38s   exit status 1
>>> MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 PATH=C:/rtools4
4/home/SYSTEM/pistache/build/src;C:/rtools44/ucrt64/bin;C:\rtools44\ucrt64\bin;C:\rtools44\usr\local
\bin;C:\rtools44\usr\bin;C:\rtools44\usr\bin;C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\progra~1\git\cmd;C:\rtools44\usr\bin\site_perl;C:\rt
ools44\usr\bin\vendor_perl;C:\rtools44\usr\bin\core_perl;\\tubfs.tu-berlin.de\kreijstal\.fzf\bin UBS
AN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 ASAN_OPTIONS=halt_on_
error=1:abort_on_error=1:print_summary=1 MESON_TEST_ITERATION=1 MALLOC_PERTURB_=138 C:/rtools44/home
/SYSTEM/pistache/build/tests/run_https_server_test.exe

28/28 pistache / listener_tls_test                OK               4.37s

Ok:                 26
Expected Fail:      0
Fail:               2
Unexpected Pass:    0
Skipped:            0
Timeout:            0

Full log written to C:/rtools44/home/SYSTEM/pistache/build/meson-logs/testlog.txt

testlog.txt

Might be interesting

[dgreatwood]

Regarding the test errors, I would take a guess that that the errors in https_server_test are due to the "wrong" version of curl being installed.

As background, certain curl commands work only when curl is using openssl and not when it is using the Windows default schannel library. I believe this is a bug in curl or schannel, but I have not proven it for certain.

In any case, to provide a fair unit test, the setup script (gccsetup.ps1 or msvcsetup.ps1) does:

($curl_openssl_there = (vcpkg list "curl[openssl]")) *> $null
if (! $curl_openssl_there) { # vcpkg list - list installed packages
    ($curl_there = (vcpkg list "curl")) *> $null
    if ($curl_there) {
        vcpkg remove curl
    }
    vcpkg install curl[openssl]
}

Could you please do these two commands:

vcpkg list "curl"
vcpkg list "curl[openssl]"

and paste the results here?

The other failure, in rest_server_test, looks unlike anything I've seen before. On inspection of rest_server_test.cpp, it appears that response.peer()->hostname() in doResolveClient is returning "view-localhost" rather than a real hostname. There are already special cases in the test to deal with response.peer()->hostname() returning "localhost" or "ip6-localhost". I guess we need to add "view-localhost" case; perhaps it is particular to this exact version of Windows. I'll create a PR to do that.

Thanks so much for your continued contribution!

[dgreatwood]

Hi @Kreijstal - I've merged what I hope is a fix for the rest_server_test issue into https://github.com/pistacheio/pistache.

When you get the chance, it would be great if you could try it out in your environment.

Thanks!

[dgreatwood]

Regarding

UnicodeEncodeError: 'charmap' codec can't encode character

According to this stackoverflow thread (see the top answer and also the comment by
xtornasol512), in Windows 10 + python3, you may see this error if writing to a file without explicitly specifying utf-8 encoding.

installman.ps1 does two write-to-files that I can see, and both do specify encoding (one ascii, the other utf-8).

What I'd suggest is - try running installman.ps1 manually from a PS prompt, and see if you can narrow down which line of installman.ps1 (if any) is causing this error to be thrown.

The only other question mark would be that you are running meson from ucrt64. In my setup, meson is installed as a Python package, with a path like this:

PS> where.exe meson.exe
[my home dir]\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts\meson.exe

Since I installed meson:
pip3 install --user meson --no-warn-script-location

I don't know if installing meson this way might make a difference.

LMK what you find...

[Kreijstal]

ahh it seems this other machine is locked up

The file
"C:\rtools44\home\SYSTEM\pistache\src\winlog\installman.ps1" cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at https://go.microsoft.com/fwlink/?LinkID=135170.

but with this #1281 (comment) patch, it removes this restriction making it compilable for low privilege windows systems.

just type

patch -p1 <<'EOF'
diff --git a/src/meson.build b/src/meson.build
index 827e523..14f4744 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -304,7 +304,7 @@ if host_machine.system() == 'windows'

 endif

-if host_machine.system() == 'windows'
+if host_machine.system() == 'windows' and compiler.get_id() == 'msvc'
     # Next we install the logging manifest and pistachelog.dll in the
     # system. We do this (install logging components alone, without
     # Pistache as a whole) as part of meson build, not just as part of
EOF

on msys2 shell.
Also these are the packages that I use

pacman -S pactoys git --needed
pacboy -S meson:p curl:p cmake:p cc:p rapidjson:p libevent:p cppcheck:p gtest:p --needed
meson setup build --buildtype=release -DPISTACHE_USE_SSL=true -DPISTACHE_BUILD_EXAMPLES=true -DPISTACHE_BUILD_TESTS=true -DPISTACHE_BUILD_DOCS=false -DPISTACHE_USE_CONTENT_ENCODING_BROTLI=true -DPISTACHE_USE_CONTENT_ENCODING_DEFLATE=true -DPISTACHE_USE_CONTENT_ENCODING_ZSTD=true --prefix="$PWD/prefix"

[dgreatwood]

ahh it seems this other machine is locked up

[DG] OK, that's interesting. You might have seen that the file Building on Windows.txt states to do:
Set-ExecutionPolicy -ExecutionPolicy Bypass
for exactly this reason.

However, if that is not possible, then as you said we need another solution.

If I may, I don't think restricting the manifest and logging.dll install to MSVC is the answer. It is just as desirable to install the manifest with gcc as with MSVC; and it is just as dependent on script execution policy.

Instead, I will look for an alternative way to allow the compile to complete successfully even if the execution policy is blocking execution of the script.

Thanks once more!
P.S. Of course, what you did is good for a workaround to allow you to work on the gcc/windmc case.

[dgreatwood]

Hi @Kreijstal -

Regarding your suggestion in #1281 (comment), instead I tried:

diff --git a/src/meson.build b/src/meson.build
index e2f2b0c..6a6772b 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -375,6 +375,7 @@ if host_machine.system() == 'windows'
           # out as part of building library('pistache'...).
       command: [powershell_prog, '-NoProfile',
                    '-NonInteractive', '-WindowStyle', 'Hidden', '-NoLogo',
+                   '-ExecutionPolicy', 'ByPass',
                    '-Command',
                    instl_log_scrpt,
                        '-dirinpstlogdll', proj_bld_rt,

At least on my machine, that extra line allowed the installman.ps1 to run as part of the build even when the CurrentUser ExecutionPolicy was set to Restricted in the registry.

And it should work the same for gcc or Visual studio cases.

Does that change work for you on your "locked down" machine?