Optimize WASM loading and improve playground examples

cs01 · cs01 · commit 2935fdabf6a6 · 2025-11-24T18:21:47.000-08:00
Performance improvements:
- Pre-load WASM binary once in main thread (60MB download)
- Share pre-loaded binary with all workers via ArrayBuffer transfer
- Eliminates redundant downloads (was 120MB+ per compile, now 0MB)
- Fresh worker per compilation still avoids signal handler buildup

UI improvements:
- Remove stats display from compiler output header for cleaner look
- Update version display to show "clang v22.0.0git" format
- Replace stdlib.h examples with self-contained code examples

New examples:
- Null Check Example: Basic flow-sensitive null checking
- Function Invalidates Check: Shows narrowing invalidation (Phase 4)
- _Nonnull Annotations: Type-level null safety enforcement

Result: Single 60MB download on first load, unlimited compilations
with zero additional downloads.
diff --git a/README.md b/README.md
@@ -39,40 +39,31 @@ Null pointer dereferences are just one category of memory safety bugs. Here's ho
 
 ### What Gets Fixed
 
-| Safety Issue | Standard C | **Null-Safe Clang** (null checking) | Clang `-fbounds-safety` | Rust |
-|-------------|------------|:-----------------------------------:|-------------------------|------|
-| Null pointer dereferences | ❌ Unsafe | **✅ Fixed** | ❌ Unsafe | ✅ Fixed |
-| Buffer overflows | ❌ Unsafe | ❌ Unsafe | ✅ Fixed | ✅ Fixed |
-| Use-after-free | ❌ Unsafe | ❌ Unsafe | ❌ Unsafe | ✅ Fixed |
-| Double-free | ❌ Unsafe | ❌ Unsafe | ❌ Unsafe | ✅ Fixed |
-| Uninitialized memory | ❌ Unsafe | ❌ Unsafe | ⚠️  Partial | ✅ Fixed |
+| Safety Issue            | Standard C | **Null-Safe Clang** (null checking) |
+|-------------------------|------------|:-----------------------------------:|
+| Null pointer dereferences | ❌ Unsafe | **✅ Fixed**                        |
+| Buffer overflows        | ❌ Unsafe  | ❌ Unsafe                            |
+| Use-after-free          | ❌ Unsafe  | ❌ Unsafe                            |
+| Double-free             | ❌ Unsafe  | ❌ Unsafe                            |
+| Uninitialized memory    | ❌ Unsafe  | ❌ Unsafe                            |
 
 
-### Why This Still Matters
+### Why you still might want to try this
 
 While Null-Safe Clang doesn't solve all memory safety issues, null pointer dereferences are a significant problem:
 
-- One in four memory safety bugs involve null pointer dereferences ([Microsoft Security Response Center](https://github.com/microsoft/MSRC-Security-Research/blob/master/papers/2019/The%20Memory%20Safety%20Story.pdf))
-- Zero runtime overhead - nullability checks are compile-time only, no runtime checks inserted
+- Many memory safety bugs involve null pointer dereferences
 - Easier to adopt than rewriting in Rust (100% compatible with existing C code)
-- Incremental deployment (warnings by default, can enable per-file)
 - Complements other efforts (combine with `-fbounds-safety` for buffer safety)
+- Incremental deployment (warnings by default, can enable per-file)
 
 
 ## Usage
 
 Basic usage (warnings enabled by default):
 ```bash
 clang mycode.c                          # Warnings for nullable dereferences
-```
-
-Promote warnings to errors:
-```bash
 clang -Werror=nullability mycode.c      # Treat nullability issues as errors
-```
-
-Disable strict nullability:
-```bash
 clang -fno-strict-nullability mycode.c  # Turn off nullability checking
 ```
 
@@ -92,15 +83,13 @@ void legacy_function(int* p) { ... }
 
 - Nullable-by-default: All pointers are `_Nullable` unless marked `_Nonnull`
 - Flow-sensitive narrowing: `if (p)` proves `p` is non-null in that scope
-- Smart invalidation: Assumes functions have side effects; use `__attribute__((pure))` or `__attribute__((const))` to preserve narrowing
 - Early-exit patterns: Understands `return`, `goto`, `break`, `continue`
-- Multi-level pointers: Works with `int**`, `int***`, etc.
 - Pointer arithmetic: `q = p + 1` preserves narrowing from `p`
-- Type checking: Through function calls, returns, and assignments
-- Typedef support: Nullability annotations work seamlessly with typedefs
+- Type checking through function calls, returns, and assignments
+- Works with Typedefs
+- Assumes functions have side effects (use `__attribute__((pure))` or `__attribute__((const))` to preserve narrowing)
 - Null-safe headers: Annotated C standard library in `clang/nullsafe-headers/`
-- IDE integration: Enhanced `clangd` with real-time nullability diagnostics
-- Real-world tested: Validated on cJSON, SQLite
+- IDE integration: `clangd` built from this fork has the same logic and warnings as clang
 
 ## How It Works
 
diff --git a/nullsafe-playground/compiler-worker.js b/nullsafe-playground/compiler-worker.js
@@ -10,7 +10,7 @@ self.onmessage = function(e) {
 
     if (type === 'load') {
         // Configure Module BEFORE loading clang.js
-        // Emscripten will check if Module exists and merge with it
+        // The main thread pre-loaded the WASM binary and is sharing it
         self.Module = {
             print: function(text) {
                 self.postMessage({ type: 'stdout', text });
@@ -19,10 +19,10 @@ self.onmessage = function(e) {
                 self.postMessage({ type: 'stderr', text });
             },
             noInitialRun: true,
+            wasmBinary: data.wasmBinary,  // Use the shared WASM binary
             postRun: [function() {
                 // postRun happens AFTER runtime initialization
                 console.log('Worker: postRun called');
-                console.log('Worker: Checking what exists on Module:', Object.keys(self.Module).filter(k => k.startsWith('FS') || k === 'FS'));
 
                 // Try different FS locations
                 const FS = self.Module.FS || self.FS || (typeof FS !== 'undefined' ? FS : null);
@@ -38,7 +38,6 @@ self.onmessage = function(e) {
                     const checkFS = setInterval(() => {
                         attempts++;
                         const fsNow = self.Module.FS || self.FS || (typeof FS !== 'undefined' ? FS : null);
-                        console.log(`Worker: Polling for FS (attempt ${attempts}):`, !!fsNow);
 
                         if (fsNow) {
                             clearInterval(checkFS);
diff --git a/nullsafe-playground/index.html b/nullsafe-playground/index.html
@@ -528,8 +528,8 @@ <h1>Null-Safe Clang Playground</h1>
 
         <select id="examplesSelect" class="examples-select">
             <option value="null-check" selected>Null Check Example</option>
-            <option value="array">Array Safety</option>
-            <option value="function">Function Parameters</option>
+            <option value="function-invalidates">Function Invalidates Check</option>
+            <option value="nonnull-annotation">_Nonnull Annotations</option>
         </select>
 
         <button id="shareBtn" class="btn btn-secondary">
@@ -564,9 +564,6 @@ <h1>Null-Safe Clang Playground</h1>
         <div class="pane">
             <div class="pane-header">
                 <span>Compiler Output</span>
-                <div class="pane-actions">
-                    <span id="outputStats" class="icon-btn" style="cursor: default;"></span>
-                </div>
             </div>
             <div class="output-container">
                 <div class="output-section">
@@ -594,7 +591,6 @@ <h1>Null-Safe Clang Playground</h1>
         const shareBtn = document.getElementById('shareBtn');
         const loadingBar = document.getElementById('loadingBar');
         const toast = document.getElementById('toast');
-        const outputStats = document.getElementById('outputStats');
         const divider = document.getElementById('divider');
         const outputDivider = document.getElementById('outputDivider');
 
@@ -608,40 +604,29 @@ <h1>Null-Safe Clang Playground</h1>
 void unsafe(int* data) {
     *data = 42;  // warning - data might be null!
 }`,
-            'array': `#include <stdlib.h>
+            'function-invalidates': `void modify(int** ptr);  // external function
 
-int main(void) {
-    int *arr = malloc(10 * sizeof(int));
+void example(int* data) {
+    if (data) {
+        *data = 1;  // OK - checked
 
-    arr[0] = 42;  // warning - malloc can return NULL
+        modify(&data);  // might modify data!
 
-    if (arr) {
-        arr[0] = 42;  // OK - arr is non-null here
-        free(arr);
+        *data = 2;  // warning - data could be null now!
     }
-
-    return 0;
 }`,
-            'function': `void process(int *data) {
-    *data = 42;  // warning - data might be null
-}
-
-void process_safe(int *data) {
-    if (data) {
-        *data = 42;  // OK - checked for null
-    }
-}
-
-void guaranteed(int * _Nonnull data) {
+            'nonnull-annotation': `// Use _Nonnull to require non-null arguments
+void process_nonnull(int * _Nonnull data) {
     *data = 42;  // OK - caller must pass non-null
 }
 
-int main(void) {
-    int x = 0;
-    process(&x);
-    process_safe(&x);
-    guaranteed(&x);
-    return 0;
+void caller(void) {
+    int* nullable = 0;
+    int x = 10;
+
+    process_nonnull(0);        // error!
+    process_nonnull(nullable); // error!
+    process_nonnull(&x);       // OK
 }`
         };
 
@@ -652,6 +637,7 @@ <h1>Null-Safe Clang Playground</h1>
         let isCompiling = false;
         let isDragging = false;
         let scriptUrl = null;  // URL for worker to load clang.js
+        let wasmBinary = null;  // Pre-loaded WASM binary to share with workers
 
         // Toast notification
         function showToast(message, duration = 2000) {
@@ -743,79 +729,40 @@ <h1>Null-Safe Clang Playground</h1>
             status.className = 'status';
             loadingBar.classList.add('active');
 
-            let capturedStdout = '';
-            let capturedStderr = '';
-
             try {
                 // Determine script URL
                 const isDev = window.location.hostname === 'localhost' || window.location.hostname === '127.0.0.1';
                 scriptUrl = isDev
                     ? './clang.js'
                     : 'https://github.com/cs01/llvm-project/releases/latest/download/clang-nullsafe.js';
 
-                // Create a temporary worker to get the version
-                const worker = new Worker('compiler-worker.js');
-
-                let versionResolved = false;
-                let loadTimeout = setTimeout(() => {
-                    if (!versionResolved) {
-                        worker.terminate();
-                        throw new Error('Compiler initialization timeout');
-                    }
-                }, 60000); // 60 second timeout
-
-                worker.onmessage = function(e) {
-                    const { type, text, error } = e.data;
-
-                    if (type === 'ready') {
-                        capturedStdout = '';
-                        capturedStderr = '';
-                        worker.postMessage({
-                            type: 'compile',
-                            code: 'int main() { return 0; }',
-                            extraFlags: ['--version']
-                        });
-                    } else if (type === 'stdout') {
-                        capturedStdout += text + '\n';
-                    } else if (type === 'stderr') {
-                        capturedStderr += text + '\n';
-                    } else if (type === 'complete') {
-                        if (!versionResolved) {
-                            versionResolved = true;
-                            clearTimeout(loadTimeout);
-
-                            const versionOutput = capturedStdout + capturedStderr;
-                            const versionMatch = versionOutput.match(/clang version ([^\s]+)/);
-                            if (versionMatch) {
-                                clangVersion = versionMatch[1];
-                                document.querySelector('.output-section-header').textContent =
-                                    `Null-Safe Clang ${clangVersion}`;
-                            }
-
-                            worker.terminate();
-
-                            status.textContent = 'Ready to compile';
-                            status.className = 'status ready';
-                            compileBtn.disabled = false;
-                            loadingBar.classList.remove('active');
-                            showToast('Compiler loaded successfully!');
-
-                            setTimeout(() => compile(), 500);
-                        }
-                    } else if (type === 'error') {
-                        clearTimeout(loadTimeout);
-                        worker.terminate();
-                        throw new Error(error);
-                    }
-                };
+                const wasmUrl = isDev
+                    ? './clang.wasm'
+                    : 'https://github.com/cs01/llvm-project/releases/latest/download/clang-nullsafe.wasm';
+
+                // Pre-load the WASM binary ONCE
+                console.log('Downloading WASM binary...');
+                const wasmResponse = await fetch(wasmUrl);
+                wasmBinary = await wasmResponse.arrayBuffer();
+                console.log('WASM binary loaded:', wasmBinary.byteLength, 'bytes');
+
+                // Now test with a worker to get the version
+                const versionResult = await compileCode('int main() { return 0; }', ['--version']);
+                const versionOutput = versionResult.stdout + versionResult.stderr;
+                const versionMatch = versionOutput.match(/clang version ([^\s]+)/);
+                if (versionMatch) {
+                    clangVersion = versionMatch[1];
+                    document.querySelector('.output-section-header').textContent =
+                        `Null-Safe Clang ${clangVersion}`;
+                }
 
-                worker.onerror = function(error) {
-                    clearTimeout(loadTimeout);
-                    worker.terminate();
-                    throw error;
-                };
+                status.textContent = 'Ready to compile';
+                status.className = 'status ready';
+                compileBtn.disabled = false;
+                loadingBar.classList.remove('active');
+                showToast('Compiler loaded successfully!');
 
-                worker.postMessage({ type: 'load', scriptUrl });
+                setTimeout(() => compile(), 500);
 
             } catch (error) {
                 status.textContent = 'Failed to load compiler';
@@ -840,33 +787,30 @@ <h1>Null-Safe Clang Playground</h1>
             status.className = 'status compiling';
             outputNullsafe.innerHTML = '';
             outputMainline.innerHTML = '';
-            outputStats.textContent = '';
             loadingBar.classList.add('active');
 
             const startTime = performance.now();
 
             try {
                 const code = editor.value;
 
-                // Compile with null-safety enabled (default)
-                const nullsafeFlags = [];
-                const nullsafeResult = await compileCode(code, nullsafeFlags);
-
-                // Compile with nullability warnings suppressed (simulates mainline)
-                const mainlineFlags = ['-Wno-nullability'];
-                const mainlineResult = await compileCode(code, mainlineFlags);
+                // Compile both versions in parallel
+                const [nullsafeResult, mainlineResult] = await Promise.all([
+                    compileCode(code, []),
+                    compileCode(code, ['-Wno-nullability'])
+                ]);
 
                 const duration = (performance.now() - startTime).toFixed(0);
 
                 // Build command strings
                 const baseArgs = '-fsyntax-only --target=wasm32-unknown-emscripten';
-                const nullsafeCmd = `$ clang ${baseArgs}${nullsafeFlags.length ? ' ' + nullsafeFlags.join(' ') : ''} input.c`;
-                const mainlineCmd = `$ clang ${baseArgs}${mainlineFlags.length ? ' ' + mainlineFlags.join(' ') : ''} input.c`;
+                const nullsafeCmd = `$ clang ${baseArgs} input.c`;
+                const mainlineCmd = `$ clang ${baseArgs} -Wno-nullability input.c`;
 
                 // Update headers with clearer titles
                 const headers = document.querySelectorAll('.output-section-header');
-                headers[0].textContent = `With Null Warnings (v${clangVersion})`;
-                headers[1].textContent = `Without Null Warnings (v${clangVersion})`;
+                headers[0].textContent = `With Null Warnings (clang v${clangVersion})`;
+                headers[1].textContent = `Without Null Warnings (clang v${clangVersion})`;
 
                 // Count nullability warnings to detect missed bugs
                 const nullWarningCount = (nullsafeResult.stderr.match(/\[-Wnullability\]/g) || []).length;
@@ -890,18 +834,6 @@ <h1>Null-Safe Clang Playground</h1>
                     }
                 }
 
-                // Calculate stats from null-safe results
-                const errorCount = (nullsafeResult.stderr.match(/error:/g) || []).length;
-                const warningCount = (nullsafeResult.stderr.match(/warning:/g) || []).length;
-
-                let stats = [];
-                if (errorCount > 0) stats.push(`${errorCount} error${errorCount !== 1 ? 's' : ''}`);
-                if (warningCount > 0) stats.push(`${warningCount} warning${warningCount !== 1 ? 's' : ''}`);
-                if (nullWarningCount > 0) stats.push(`(${nullWarningCount} null bug${nullWarningCount !== 1 ? 's' : ''})`);
-                stats.push(`${duration}ms`);
-
-                outputStats.textContent = stats.join(' • ');
-
                 status.textContent = 'Ready to compile';
                 status.className = 'status ready';
             } catch (error) {
@@ -910,7 +842,6 @@ <h1>Null-Safe Clang Playground</h1>
                 outputMainline.innerHTML = `<span class="error">Compilation failed: ${errorMsg}\n\nCheck console for details.</span>`;
                 status.textContent = 'Compilation error';
                 status.className = 'status error';
-                outputStats.textContent = '✗ Failed';
                 console.error('Compilation error:', error);
             } finally {
                 isCompiling = false;
@@ -938,6 +869,7 @@ <h1>Null-Safe Clang Playground</h1>
                     const { type, text, error, exitCode } = e.data;
 
                     if (type === 'ready') {
+                        // Worker is ready, send compile request
                         worker.postMessage({
                             type: 'compile',
                             code,
@@ -967,7 +899,12 @@ <h1>Null-Safe Clang Playground</h1>
                     reject(error);
                 };
 
-                worker.postMessage({ type: 'load', scriptUrl });
+                // Send the pre-loaded WASM binary to the worker
+                worker.postMessage({
+                    type: 'load',
+                    scriptUrl,
+                    wasmBinary  // Share the pre-loaded WASM
+                });
             });
         }
 
