Skip to content

QVAC-8561: Merge DL and other progress from 6469 #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 17 commits into from
Closed

QVAC-8561: Merge DL and other progress from 6469 #69

wants to merge 17 commits into from

Conversation

@jesusmb1995
Copy link

@jesusmb1995 jesusmb1995 commented Nov 27, 2025

jesusmb1995 and others added 17 commits November 27, 2025 10:45
@jesusmb1995
default_dl_search
a589b3a
@jesusmb1995
log
2329deb
@jesusmb1995
force_SHARD_LIBSON
1d8f07d
@jesusmb1995
adreno_logic
a11812d
@jesusmb1995
optimizedCpuEntries
b846c3b
@jesusmb1995
backend_selection
3707487
@jesusmb1995
disable_ci_test_for_cmake
e40ecbb
@jesusmb1995
disable_flaky_shift_disabled_short_prompt
150efd6
@jesusmb1995
disable_tests_failing_repositories
f4eb93a
@jesusmb1995
supress_test_sanitizer
b2c54cc 
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 87edff8..559bda344 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -291,6 +291,15 @@ jobs:

       - name: Test
         id: cmake_test
+        env:
+          # AddressSanitizer options
+          ASAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stats=1:check_initialization_order=1:strict_init_order=1:detect_stack_use_after_return=1:print_summary=1:print_scariness=1:print_legend=1"
+          # ThreadSanitizer options
+          TSAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stats=1:print_summary=1:print_legend=1"
+          # UndefinedBehaviorSanitizer options
+          UBSAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stacktrace=1:print_summary=1"
+          # Common options for all sanitizers
+          MSAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stats=1"
         run: |
           cd build
           ctest -L main --verbose --timeout 900
@@ -921,6 +930,15 @@ jobs:
       - name: Test
         id: cmake_test
         if: ${{ matrix.arch == 'x64' }}
+        env:
+          # AddressSanitizer options
+          ASAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stats=1:check_initialization_order=1:strict_init_order=1:detect_stack_use_after_return=1:print_summary=1:print_scariness=1:print_legend=1"
+          # ThreadSanitizer options
+          TSAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stats=1:print_summary=1:print_legend=1"
+          # UndefinedBehaviorSanitizer options
+          UBSAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stacktrace=1:print_summary=1"
+          # Common options for all sanitizers
+          MSAN_OPTIONS: "verbosity=1:abort_on_error=1:print_stats=1"
         run: |
           cd build
           ctest -L main -C Release --verbose --timeout 900
diff --git a/tests/test-gguf.cpp b/tests/test-gguf.cpp
index 3f0c312..1c852934c 100644
--- a/tests/test-gguf.cpp
+++ b/tests/test-gguf.cpp
@@ -101,6 +101,24 @@ static bool expect_context_not_null(const enum handcrafted_file_type hft) {

 typedef std::pair<enum ggml_type, std::array<int64_t, GGML_MAX_DIMS>> tensor_config_t;

+// Helper function to safely cast to gguf_type, suppressing sanitizer warnings for intentional invalid values
+// Portable implementation for disabling sanitizer attributes, depending on compiler
+#if defined(__clang__) || defined(__GNUC__)
+static inline enum gguf_type __attribute__((no_sanitize("undefined")))
+safe_cast_to_gguf_type(int value) {
+    return static_cast<enum gguf_type>(value);
+}
+#elif defined(_MSC_VER)
+// MSVC does not support __attribute__; just define without it
+static inline enum gguf_type safe_cast_to_gguf_type(int value) {
+    return static_cast<enum gguf_type>(value);
+}
+#else
+static inline enum gguf_type safe_cast_to_gguf_type(int value) {
+    return static_cast<enum gguf_type>(value);
+}
+#endif
+
 static std::vector<tensor_config_t> get_tensor_configs(std::mt19937 & rng) {
     std::vector<tensor_config_t> tensor_configs;
     tensor_configs.reserve(100);
@@ -140,7 +158,9 @@ static std::vector<std::pair<enum gguf_type, enum gguf_type>> get_kv_types(std::
             continue;
         }

-        kv_types.push_back(std::make_pair(type, gguf_type(-1)));
+        // Intentionally create invalid enum value for testing error handling
+        // Suppress sanitizer warning as this is intentional undefined behavior for testing
+        kv_types.push_back(std::make_pair(type, safe_cast_to_gguf_type(-1)));
     }
     std::shuffle(kv_types.begin(), kv_types.end(), rng);

@@ -232,8 +252,10 @@ static FILE * get_handcrafted_file(const unsigned int seed, const enum handcraft
     }

     for (int i = 0; i < int(kv_types.size()); ++i) {
-        const enum gguf_type type     = gguf_type(hft == HANDCRAFTED_KV_BAD_TYPE ? GGUF_TYPE_COUNT : kv_types[i].first);
-        const enum gguf_type type_arr = gguf_type(hft == HANDCRAFTED_KV_BAD_TYPE ? GGUF_TYPE_COUNT : kv_types[i].second);
+        // Intentionally create invalid enum values for testing error handling
+        // Suppress sanitizer warning as this is intentional undefined behavior for testing
+        const enum gguf_type type     = safe_cast_to_gguf_type(hft == HANDCRAFTED_KV_BAD_TYPE ? GGUF_TYPE_COUNT : kv_types[i].first);
+        const enum gguf_type type_arr = safe_cast_to_gguf_type(hft == HANDCRAFTED_KV_BAD_TYPE ? GGUF_TYPE_COUNT : kv_types[i].second);

         const std::string key = "my_key_" + std::to_string((hft == HANDCRAFTED_KV_DUPLICATE_KEY ? i/2 : i));

@@ -463,8 +485,9 @@ static bool handcrafted_check_kv(const gguf_context * gguf_ctx, const unsigned i
     bool ok = true;

     for (int i = 0; i < int(kv_types.size()); ++i) {
-        const enum gguf_type type     = gguf_type(kv_types[i].first);
-        const enum gguf_type type_arr = gguf_type(kv_types[i].second);
+        // Suppress sanitizer warning for intentional invalid enum values in test data
+        const enum gguf_type type     = safe_cast_to_gguf_type(kv_types[i].first);
+        const enum gguf_type type_arr = safe_cast_to_gguf_type(kv_types[i].second);

         const std::string key = "my_key_" + std::to_string(i);
@jesusmb1995
fix_flake8_linter
5a14139
@jpgaribotti @jesusmb1995
Merge pull request ggml-org#52 from jesusmb1995/jmb/build_dl15
6f8d0bf 
QVAC-7519: Dynamic loading of backends
@jesusmb1995
fix_absolute_install
aad8f15
@jpgaribotti @jesusmb1995
Merge pull request ggml-org#57 from jesusmb1995/jmb/build_dl18
d556752 
QVAC-7519: Fix dynamic libs path with vcpkg install (issue during cached CI)
@jesusmb1995
fix_absolute_install
aa43821
@jesusmb1995
remove_cache
c335c87
@jpgaribotti @jesusmb1995
Merge pull request ggml-org#59 from jesusmb1995/jmb/build_dl20
5cd6c4f 
 QVAC-7519: Fix dynamic libs path with vcpkg install
@jesusmb1995 jesusmb1995 self-assigned this Nov 27, 2025
@jesusmb1995
Copy link
Author

jesusmb1995 commented Nov 27, 2025

No need to merge this PR, most changes seem to be already on temp-latest. There are new tests failing on CI though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@jesusmb1995 jesusmb1995

Labels

devops

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jesusmb1995 @jpgaribotti