Skip to content

Standard Library Modules Bug Bash

Jump to bottom
Stephan T. Lavavej edited this page Jul 21, 2022 · 37 revisions

Overview

About

I'm working on the Standard Library Modules feature. It's really exciting, as it'll make code faster to compile and simpler to write - you'll be able to import std; instead of including dozens of headers. I've already gone through over 145,000 lines of the STL's headers and manually added over 3,700 exports. One of the major challenges is that C++ is a complex language, the STL uses it in complex ways, and digesting that usage into a new serialized format has revealed many corner cases in the compiler. I've already reported over 50 compiler bugs while working on this (and the earlier Standard Library Header Units, which uses much of the same compiler machinery). Almost all of those compiler bugs have been fixed thanks to the compiler front-end team's tireless work, so compiler stability has greatly improved, but more work remains to be done.

That's where this bug bash comes in. I want Standard Library Modules to ship with a very high level of quality, so users will have a great experience and can begin modularizing their codebases. I've already tested some machinery from every header, but I'm sure there are more bugs lurking that will be revealed by more usage. So, I'm inviting you to try out a branch from my personal fork, and report any issues you encounter that are specific to Standard Library Modules versus classic includes. I'll take care of the rest (analyzing them to see whether the compiler or the library needs to fixed, and whether a library workaround is possible).

Links

  • P2465R3 Standard Library Modules std and std.compat
    • This is the latest version of the proposal, which is scheduled for a formal vote on 2022-07-25.
  • N4910 C++ Working Draft
    • This is the latest version of the C++23 Working Draft, released 2022-03-17.
  • MSVC Compiler Options
    • This explains /MTd, /Od, etc.
  • microsoft/STL#1694 Standard Library Header Units and Modules - tracking issue
    • This records the many compiler bugs that I've encountered and reported while working on this feature.

Getting Started

Install VS Preview

  1. Install the latest VS Community 2022 Preview.
    • You must use the latest Preview (currently 17.3 Preview 4), which contains important compiler bugfixes. The latest production version (currently 17.2) will not work for this bug bash.
    • The edition doesn't matter, if you happen to already have Professional or Enterprise installed.
  2. The VS Installer will display the Workloads tab. Scroll down and select "Desktop development with C++".
  3. In the "Installation details" pane to the right, deselect the Windows 10 SDK (10.0.19041.0).
    • The STL's build will reject this version.
  4. Scroll down and select the Windows 11 SDK (10.0.22000.0).
    • The STL's build requires this version, but your OS doesn't need to be Windows 11; Windows 10 should work fine.
    • We haven't tested the newer version of the Windows 11 SDK (10.0.22621.0) when building the STL, although in theory it should work.
  5. Begin the installation.
    • The installer should eventually say "Visual Studio has been successfully installed. We recommend rebooting soon to clean up any remaining files."
  6. Reboot.

Clone the STL

These are quick-start instructions for the bug bash, not meant for normal STL development.

  1. Install git, if you haven't already.
    • If you're unfamiliar with the options, you can accept the defaults, although I have two recommendations:
      • Select a text editor you're comfortable with.
      • Select "Checkout as-is, commit as-is" for line endings.
  2. Open an "x64 Native Tools Command Prompt for VS 2022 Preview".
  3. Change to a working directory where you'd like to clone the STL. In this example, I'll use C:\GitHub.
  4. git clone https://github.com/StephanTLavavej/STL --branch bug-bash
  5. cd STL
  6. git submodule update --init --progress boost-math

Build the STL

  1. cmake -G Ninja -S . -B out\x64 -DBUILD_TESTING=OFF
  2. ninja -C out\x64
  3. out\x64\set_environment.bat
    • After a successful build, set_environment.bat adds the freshly-built STL to the INCLUDE and LIB environment variables so it can be used.

Build and Test the Modules

  1. cl /EHsc /nologo /W4 /std:c++latest /MTd /Od /c stl\modules\std.ixx stl\modules\std.compat.ixx
  2. cl /EHsc /nologo /W4 /std:c++latest /MTd /Od meow.cpp std.obj
  3. meow.exe

Congratulations - you're now one of the first users of import std;! 🎉 😸

Expected Output

Click to expand expected output: 
C:\GitHub>git clone https://github.com/StephanTLavavej/STL --branch bug-bash
Cloning into 'STL'...
remote: Enumerating objects: 21009, done.
remote: Counting objects: 100% (558/558), done.
remote: Compressing objects: 100% (242/242), done.
remote: Total 21009 (delta 366), reused 451 (delta 314), pack-reused 20451
Receiving objects: 100% (21009/21009), 15.61 MiB | 20.08 MiB/s, done.
Resolving deltas: 100% (14484/14484), done.
Updating files: 100% (2143/2143), done.

C:\GitHub>cd STL

C:\GitHub\STL>git submodule update --init --progress boost-math
Submodule 'boost-math' (https://github.com/boostorg/math.git) registered for path 'boost-math'
Cloning into 'C:/GitHub/STL/boost-math'...
remote: Enumerating objects: 104107, done.
remote: Counting objects: 100% (6758/6758), done.
remote: Compressing objects: 100% (2336/2336), done.
remote: Total 104107 (delta 4367), reused 6275 (delta 3979), pack-reused 97349
Receiving objects: 100% (104107/104107), 179.86 MiB | 30.09 MiB/s, done.
Resolving deltas: 100% (79338/79338), done.
Submodule path 'boost-math': checked out '88c4b8d661dbe15be7317f04f1eccae0c9c39980'

C:\GitHub\STL>cmake -G Ninja -S . -B out\x64 -DBUILD_TESTING=OFF
-- The CXX compiler identification is MSVC 19.33.31627.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Preview/VC/Tools/MSVC/14.33.31627/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test WINDOWS_SDK_VERSION_CHECK
-- Performing Test WINDOWS_SDK_VERSION_CHECK - Success
-- The ASM_MASM compiler identification is MSVC
-- Found assembler: C:/Program Files/Microsoft Visual Studio/2022/Preview/VC/Tools/MSVC/14.33.31627/bin/Hostx64/x64/ml64.exe
-- Searching for VS clang-format
-- Searching for VS clang-format - found
-- Boost.Math: standalone mode ON
-- Configuring done
-- Generating done
-- Build files have been written to: C:/GitHub/STL/out/x64

C:\GitHub\STL>ninja -C out\x64
ninja: Entering directory `out\x64'
[1/1014] Building ASM_MASM object stl\CMakeFiles\stl_alias_objects.dir\src\alias_init_once_begin_initialize.asm.obj
Microsoft (R) Macro Assembler (x64) Version 14.33.31627.1
Copyright (C) Microsoft Corporation.  All rights reserved.

 Assembling: C:\GitHub\STL\stl\src\alias_init_once_begin_initialize.asm
[2/1014] Building ASM_MASM object stl\CMakeFiles\stl_alias_objects.dir\src\alias_init_once_complete.asm.obj
Microsoft (R) Macro Assembler (x64) Version 14.33.31627.1
Copyright (C) Microsoft Corporation.  All rights reserved.

 Assembling: C:\GitHub\STL\stl\src\alias_init_once_complete.asm
[1014/1014] Linking CXX static library out\lib\amd64\libcpmtd0.lib

C:\GitHub\STL>out\x64\set_environment.bat

C:\GitHub\STL>cl /EHsc /nologo /W4 /std:c++latest /MTd /Od /c stl\modules\std.ixx stl\modules\std.compat.ixx
std.ixx
std.compat.ixx
Generating Code...

C:\GitHub\STL>cl /EHsc /nologo /W4 /std:c++latest /MTd /Od meow.cpp std.obj
meow.cpp

C:\GitHub\STL>meow.exe
Hello, modules world!
11 22 33
Testing <algorithm>.
Testing <any>.
Testing <array>.
Testing <atomic>.
Testing <barrier>.
Testing <bit>.
Testing <bitset>.
Testing <charconv>.
Testing <chrono>.
Testing <codecvt>.
Testing <compare>.
Testing <complex>.
Testing <concepts>.
Testing <condition_variable>.
Testing <coroutine>.
Testing <deque>.
Testing <exception>.
Testing <execution>.
Testing <expected>.
Testing <filesystem>.
Testing <format>.
Testing <forward_list>.
Testing <fstream>.
Testing <functional>.
Testing <future>.
Testing <initializer_list>.
Testing <iomanip>.
Testing <ios>.
Testing <iosfwd>.
Testing <iostream>.
Cute fluffy kittens!
Testing <istream>.
Testing <iterator>.
Testing <latch>.
Testing <limits>.
Testing <list>.
Testing <locale>.
Testing <map>.
Testing <memory_resource>.
Testing <memory>.
Testing <mutex>.
Testing <new>.
Testing <numbers>.
Testing <numeric>.
Testing <optional>.
Testing <ostream>.
Testing <queue>.
Testing <random>.
Testing <ranges>.
Testing <ratio>.
Testing <regex>.
Testing <scoped_allocator>.
Testing <semaphore>.
Testing <set>.
Testing <shared_mutex>.
Testing <source_location>.
Testing <span>.
Testing <spanstream>.
Testing <sstream>.
Testing <stack>.
Testing <stacktrace>.
meow+0x6184
Testing <stdexcept>.
Testing <stop_token>.
Testing <streambuf>.
Testing <string_view>.
Testing <string>.
Testing <strstream>.
Testing <syncstream>.
Kittens are cute and fluffy!
Testing <system_error>.
Testing <thread>.
Testing <tuple>.
Testing <type_traits>.
Testing <typeindex>.
Testing <typeinfo>.
Testing <unordered_map>.
Testing <unordered_set>.
Testing <utility>.
Testing <valarray>.
Testing <variant>.
Testing <vector>.
Skipping <version> (nothing to do, macros unavailable).
Done!

Classic Includes vs. Named Modules

Success

Here's an example where they both work:

C:\GitHub\STL>type purr.cpp

#include <assert.h>

#ifdef USE_CLASSIC_INCLUDES
    #include <cstdio>
    #include <vector>
#else
    import std;
#endif

int main() {
#ifdef USE_CLASSIC_INCLUDES
    std::puts("Classic includes.");
#else
    std::puts("Named modules.");
#endif

    std::vector<int> vec{11, 22, 33};
    assert(vec[1] == 22);

    std::printf("vec: ");
    for (const auto& elem : vec) {
        std::printf("%d ", elem);
    }
    std::printf("\n");
}
C:\GitHub\STL>cl /EHsc /nologo /W4 /std:c++latest /MTd /Od purr.cpp /DUSE_CLASSIC_INCLUDES && purr.exe
purr.cpp
Classic includes.
vec: 11 22 33

C:\GitHub\STL>cl /EHsc /nologo /W4 /std:c++latest /MTd /Od purr.cpp std.obj && purr.exe
purr.cpp
Named modules.
vec: 11 22 33

Failure

Here's an example where classic includes work, but named modules emit a compiler error:

C:\GitHub\STL>type woof.cpp

#include <assert.h>

#ifdef USE_CLASSIC_INCLUDES
    #include <cerrno>
    #include <cmath>
    #include <cstdio>
#else
    import std;
#endif

int main() {
#ifdef USE_CLASSIC_INCLUDES
    std::puts("Classic includes.");
#else
    std::puts("Named modules.");
#endif

    const double dbl{std::sqrt(2.25)};
    const int err{errno};
    std::printf("dbl: %g\n", dbl);
    std::printf("err: %d\n", err);
}
C:\GitHub\STL>cl /EHsc /nologo /W4 /std:c++latest /MTd /Od woof.cpp /DUSE_CLASSIC_INCLUDES && woof.exe
woof.cpp
Classic includes.
dbl: 1.5
err: 0

C:\GitHub\STL>cl /EHsc /nologo /W4 /std:c++latest /MTd /Od woof.cpp std.obj && woof.exe
woof.cpp
woof.cpp(19): error C2065: 'errno': undeclared identifier

This happens to be by design (named modules don't emit macros). Aside from that, this would be a perfect bug report - it clearly shows that the code works with #include and fails with import std;. It also shows the entire code, command lines, and compiler output.

Clone this wiki locally