Remove the size limit for memory read and write (revamped) #2144
Conversation
…m_t and uc_read_mem_t
…ve-size-limit-mem-read-write
|
Oh, I just noticed the github workflow for zig. We might need to extract the modifications to the build.zig.zon file then. The action included for Zig is goto-bus-stop setup-zig, which recently has become unmaintained. |
|
Before starting a review, I would like to know if it is a full re-implementation? If you still borrow some code from him, I will add Co-Author-By to git message when accepting the PR. |
|
Please add @secretnonempty as a coauthor. I simply modified his changes to the current state of the project, which has somewhat changed due to some refactorings. I will start wrangling with the pipeline. As introduced in the PR text, the android system sadly reached an out of memory condition, which resulted in a kernel panic. That will be fun to mitigate. |
…to get access to current zig releases
…avoid infinite low memory killer loop
|
ping me when you feel it ready to review
…________________________________
From: Richard Liebig ***@***.***>
Sent: Sunday, April 6, 2025 5:45:54 PM
To: unicorn-engine/unicorn ***@***.***>
Cc: lazymio ***@***.***>; Comment ***@***.***>
Subject: Re: [unicorn-engine/unicorn] Remove the size limit for memory read and write (revamped) (PR #2144)
Please add @secretnonempty<https://github.com/secretnonempty> as a coauthor. I simply modified his changes to the current state of the project, which has somewhat changed due to some refactorings. I will start wrangling with the pipeline. As introduced in the PR text, the android system sadly reached an out of memory condition, which resulted in a kernel panic. That will be fun to mitigate.
―
Reply to this email directly, view it on GitHub<#2144 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHJULO5WE7N3J57N2ZYPHVL2YDZVDAVCNFSM6AAAAAB2Q33JYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBRGMZTKMJYGY>.
You are receiving this because you commented.Message ID: ***@***.***>
[rliebig]rliebig left a comment (unicorn-engine/unicorn#2144)<#2144 (comment)>
Please add @secretnonempty<https://github.com/secretnonempty> as a coauthor. I simply modified his changes to the current state of the project, which has somewhat changed due to some refactorings. I will start wrangling with the pipeline. As introduced in the PR text, the android system sadly reached an out of memory condition, which resulted in a kernel panic. That will be fun to mitigate.
―
Reply to this email directly, view it on GitHub<#2144 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHJULO5WE7N3J57N2ZYPHVL2YDZVDAVCNFSM6AAAAAB2Q33JYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBRGMZTKMJYGY>.
You are receiving this because you commented.Message ID: ***@***.***>
|
…use there cannot be larger vec![] anyway
…ite to correct types
|
@wtdcode I believe I am there. CI/CD Tests are passing and manual tests for my use case work. |
|
The quality of the PR is rather high and I appreciate your efforts. I don't find anything to comment at this moment and I think this can be accepted as it is. However, I need to deal with #2138 firstly so that the following PR won't conflict with that. By the way, if I understand it correctly, it looks like it only introduces break changes on 32 bits platforms where |
|
I believe it is slightly more complicated: QEMU natively supports int64 values(as Now, in the case of 32bit |
|
I understand the semantics and the issues to fix of both pull requests. What I said "break" only refers to the API signatures of our current C API defined in |
|
Sorry, my bad! Yes, that should be the case. |
|
Hello, we have adopted #2138 . Could you fix the conflicts? Once that is done, I think it’s ready to get merged. Thanks in advance! |
|
Sure, will ping you when I am Done! |
|
@wtdcode at least on my end CI/CD tests are passing - could you run them here please? |
|
Thanks and welcome! |
…ngine#2144) * modify uc_priv.h to use 64 bit hwaddr type for length for uc_write_mem_t and uc_read_mem_t * propogate 64 bit address support to memory function inside unicorn.h * modify physical memory functions to ensure 64 bit support for size parameters * apply uint64_t size handling inside the memory handling main functions * remove INT_MAX size check as we now use a 64 bit address * add @secretnonempty unit test for large memory addresses to avoid regression * replace size_t in python3 bindings with u64 * adjust rust bindings * use format.sh script to improve formatting * small adjustments to java bindings, because jlong type is sometimes already used * adjust dotnet bindings to reflect new type sizes * adjust pascal bindings * adjust ruby bindings * adjust vb6 bindings * adjust zig bindings for 64 bit size support * fix build.zig.zon for newer zig versions * add haskell binding support for 64 bit sizes * use mlugg/setup-zig instead of goto-bus-stop/setup-zig github action to get access to current zig releases * use uint64_t instead of size_t for uc_mem_unmap * add whitespace between functions to preserver formatting * add precompiler check for android and abort execution of testcase to avoid infinite low memory killer loop * downgrade zig version to 0.14.0 again to ensure that pipeline can run through * adjust rust mmio_unwrap type to use u64 * change ffi size type of uc_mem_map_ptr into u64 * add warning to rust slicing * return read_mem_as_vec to earlier state, leave usize as argument because there cannot be larger vec![] anyway * adjust regions inside MmioCallbackScope to use u64 for region sizes * adjust regions inside MmioCallbackScope to use u64 for region sizes inside search vec * finalize dotnet bindings inside IBinding.fs * fix typo * adjust types inside Unicorn.fs * finalize type wrangling inside F# bindings, setting MemRead and MemWrite to correct types * fix missing bracket inside bindings/rust/src/lib.rs * replace ffi interface to hook interface inside rust lib * fix testcases to use u64 for longer references Co-authored-by: r0b0t <[email protected]>
This PR is a reimplementation of PR #1799 made by @secretnonempty, is made mainly created for technical reasons:
I do not want to attribute to any of the great work done by @secretnonempty, I am simply interested in having these changes in the upcoming release.
Some modifications had to be made, some of these were necessary to reflect the changes done during the stale period.
Please be aware that the originial test made by @secretnonempty allocates a least 4GB of memory, which can strain some systems and might invoke a
SIGKILLif there is not enough memory available. The writing operations inside the large write operations are also preformed consecutively, I think yielding the thread somewhere in between the write operation could be useful for large writes to avoid system freezes.I also added the handling to the Zig and Haskell bindings.
Zig seems only to correctly build with version 0.13.0, but not 0.14.0 Currently, the main release branch of Ziglang is at 0.15.0, althrough my interest in that part is somewhat limited. These changes are related to Zig Issue 20178, so I took some liberty to adjust it to work, so I can at least test these.
I might also add a new test case for the specific problem, which sent me done this specific rabbit hole. It was related to a large call for
mprotect.I really would like some tests for these different kind of language bindings, seeing as I had to touch quite a bit of code over several files.