Merged
Conversation
added 18 commits
March 31, 2025 08:12
The test runner checks the sourcify manifest for a list of all shards for a particular chain, then downloads each sequentially, unpacks, and finally tests each source file. This is a very basic runner and there are a lot of improvements that can be made, plus a lot of missing features. But this is a pretty good start.
…imports * Reuse a buffer when reading source files for efficiency * Reorganize code
…e parallelized. Now they are using proper iterators, and `Contract::Item` is a struct which contains a file handle instead of the file contents. This means that we can still use a shared buffer to read all of the files, improving performance, without having to borrow the string through several layers of iterators. This was what prompted me to move away from "real" iterators previously. Like the sanctuary tests, I'm using `rayon` to parallize the runner. Here, contracts are processed in parallel, but the source files within a single contract are processed sequentially. Most contracts don't have a large number of source files and trying to parallelize them as well would be unnecessary complexity.
…t them "in the background". This means less time waiting in-between shards, which matters more now that processing each shard is internally parallelized. There could still be some improvements here, specifically with making the fetches truly async. Right now we're using `reqwest`, which requires `tokio` to be used as the async runtime. It's possible that we could find a different library that would allow us to make async calls using only the `futures` crate, which would be much better suited to our use case.
…of how imports/files are resolved when building a compilation unit, since the previous way wasn't working in practice.
… specify a specific contract to test
* Allow the user to not include partial_match contracts. They are still included by default * Categorize contracts between full_match and partial_match
…ning up confusing code. * Fetch archives in the main thread, and process them in a separate thread. This means that the thread doesn't have to take ownership of the `Repository` instance, and so that will get dropped at the correct time. * Following that change, moved the fetching logic into a closure that we invoke immediately. This is all so that `tx` (Sender) can be dropped before calling `process_thread.join()`. Otherwise, the processing thread will get stuck waiting for a new message forever and will never be joined. * Removing all of the custom iterators that I built for `ContractArchive` and `Contract`. The `Contract` iterators were no longer being used since I started building compilation units by traversing the import tree. The `ContractArchive` iterator was replaced by a function `ContractArchive::contracts`, which returns an `impl Iterator`. I thought that this would be a much simpler way to express this logic, especially since the other custom iterator was removed. * Adding additional terminal output to inform the user about the progress of the test runner.
* Emit events report after testing is complete
|
Contributor
Author
|
Just ran a parsing test locally on the entire Ethereum mainnet. Here are the high-level stats:
|
added 7 commits
April 16, 2025 10:33
… Sanctuary tests. There are 4 known ones in the Ethereum mainnet, but right now I only have the address for 1
… when there is no context
* Add GitHub workflow for sourcify tests
OmarTawfik
requested changes
Apr 29, 2025
Contributor
OmarTawfik
left a comment
OmarTawfik
left a comment
There was a problem hiding this comment.
Thanks for working on this!
OmarTawfik
reviewed
May 5, 2025
OmarTawfik
reviewed
May 5, 2025
Co-authored-by: Omar Tawfik <15987992+OmarTawfik@users.noreply.github.com>
…o try to test the workflow. This trigger should be removed before merging because we don't actually want it to run on every PR.
OmarTawfik
approved these changes
May 5, 2025
Contributor
OmarTawfik
left a comment
OmarTawfik
left a comment
There was a problem hiding this comment.
Left a couple of comments. But otherwise LGTM.
Thanks!
added 14 commits
May 5, 2025 09:11
* Test workflow inputs with quoted strings instead of "raw" strings
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run e2e tests on real-world contracts using the Sourcify dataset. This will eventually replace the Sanctuary tests.
Still to-do: