Provide a mechanism to write tests around protected/guarded memory

[Dimension4]

This is a continuation of a discussion that started in the C# Discord, but let me restate the original problem:

I have a method Foo that does some SIMD and pointer shenanigans. Now I want to write tests for it. One thing I'd like to test is out of bounds accesses of the data I process, for example vector loads/stores that go beyond the input/output buffer.

I tried to set up three contiguous pages [A B C] using VirtualAlloc and then protect A and C with VirtualProtect by applying PAGE_GUARD to them. The idea is to place my input/output buffers in B, so that if an out of bounds access happens, I can either catch an exception or just let it happen and do nothing. After Foo ran, I can inspect A and C with VirtualLock to see if both pages are still guarded. If not, that means I accidentally ran over one of the buffers and accessed data I shouldn't have. In this scenario I could simply fail the test and then fix the bug in my implementation.

Removing some boilerplate, this is what the test would look like:

unsafe void FooTest()
{
    const int pageSize = 4096;

    void* memory = PInvoke.VirtualAlloc(null, (nuint) pageSize * 3, VIRTUAL_ALLOCATION_TYPE.MEM_RESERVE | VIRTUAL_ALLOCATION_TYPE.MEM_COMMIT, PAGE_PROTECTION_FLAGS.PAGE_READWRITE);
    void* lowerGuard = memory;
    void* usableMemory = (byte*) memory + pageSize;
    void* upperGuard = (byte*) memory + pageSize * 2;

    PInvoke.VirtualProtect(lowerGuard, 1, PAGE_PROTECTION_FLAGS.PAGE_READWRITE | PAGE_PROTECTION_FLAGS.PAGE_GUARD, out _);
    PInvoke.VirtualProtect(upperGuard, 1, PAGE_PROTECTION_FLAGS.PAGE_READWRITE | PAGE_PROTECTION_FLAGS.PAGE_GUARD, out _);

    var data = new Span<byte>(usableMemory, pageSize);
    Foo(data);

    bool stillGuarded = PInvoke.VirtualLock(lowerGuard, 1) && PInvoke.VirtualLock(lowerGuard, 1);
    Assert(stillGuarded);

    _ = PInvoke.VirtualFree(memory, 0, VIRTUAL_FREE_TYPE.MEM_RELEASE);
}

Unfortunately, I cannot quite implement a test like this. The problem is that the CLR catches these structured exceptions and shuts down the process. Depending on how I setup the pages, I get either an AccessViolationException or ExecutionEngineException, both of which cannot be recovered from. This means, my test outcome can only be success or crash, which is not ideal (especially when running in a CI pipeline).

I have not found a better way to test for this. I could catch out of bound writes by simply having extra space around my output buffer and check if something has been written to that space, but I don't know how to catch out of bound reads on my input buffer.

I also tried to have a C wrapper that handles these exceptions, but since it has to call back into the runtime, the runtime is still the first to catch the exception:

__declspec(dllexport) int CallWithSehHandling(Callback callback, void* context) // will be called from FooTest
{
    __try
    {
        callback(context); // calls back into C# to execute Foo
        return 0;
    }
    __except (ExceptionHandler(GetExceptionCode())) // catch EXCEPTION_GUARD_PAGE
    {
        return 1;
    }
}

I know this approach isn't flawless and I could still get legitimate AVEs by dereferencing a garbage pointer. That's not the case here though. I just have a range of memory that I try to not iterate out of.

In the end I just need a mechanism to ensure that out of bounds accesses can be handled in a graceful way. Or even better, have something that only allows memory access within a certain region within some context, but I doubt that's doable.

Any chances something like this can be supported by the runtime in the future?