-
Notifications
You must be signed in to change notification settings - Fork 275
feat: gethclone
binary
#1215
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ARR4N
wants to merge
9
commits into
master
Choose a base branch
from
arr4n/gethclone
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
feat: gethclone
binary
#1215
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
7bdba85
feat: recursively find all `geth` deps of a set of root packages
ARR4N bc958b4
feat: add additional copyright + generated-code comment
ARR4N 6d4c0d2
feat: replace all `github.com/ethereum/go-ethereum` import paths
ARR4N 7044686
fix: use geth import already in `scripts/geth-allowed-packages.txt`
ARR4N f67e79a
feat: AST patching w/ package+type matching
ARR4N 59fc25f
refactor: abstract `astpatch` package
ARR4N 4529a3b
fix: false-positive geth import in linter
ARR4N df5d9e0
feat: accept a `go.mod` file to describe output destination
ARR4N 8df8e09
refactor: `config` and methods into separate file for minimal entrypo…
ARR4N File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Experimental | ||
|
||
Code in this directory is experimental and MUST NOT be relied on to be stable. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# `gethclone` | ||
|
||
This is an experimental module for tracking upstream `go-ethereum` changes. | ||
The approach of `git merge`ing the upstream branch into `subnet-evm` is brittle as it relies on purely syntactic patching. | ||
`gethclone` is intended to follow a rebase-like pattern, applying a set of semantic patches (e.g. AST modification, [Uber's `gopatch`](https://pkg.go.dev/github.com/uber-go/gopatch), etc.) that (a) should be more robust to refactoring; and (b) act as explicit documentation of the diffs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,91 @@ | ||
// Package astpatch provides functionality for traversing and modifying Go | ||
// syntax trees. It extends the astutil package with reusable "patches". | ||
package astpatch | ||
|
||
import ( | ||
"go/ast" | ||
"reflect" | ||
|
||
"golang.org/x/tools/go/ast/astutil" | ||
) | ||
|
||
type ( | ||
// A Patch (optionally) modifies an AST node; it is equivalent to an | ||
// `astutil.ApplyFunc` except that it returns an error instead of a boolean. | ||
// A non-nil error is equivalent to returning false and will also abort all | ||
// further calls to other patches. | ||
Patch func(*astutil.Cursor) error | ||
// A PatchRegistry maps [Go package path] -> [ast.Node concrete types] -> | ||
// [all `Patch` functions that must be applied to said node types in said | ||
// package]. | ||
// | ||
// The special `pkgPath` value "*" will match all package paths. | ||
PatchRegistry map[string]map[reflect.Type][]Patch | ||
) | ||
|
||
// Add is a convenience wrapper for registering a new `Patch` in the registry. | ||
// The `zeroNode` can be any type (including nil pointers) that implements | ||
// `ast.Node`. | ||
// | ||
// The special `pkgPath` value "*" will match all package paths. While there is | ||
// no specific requirement for `pkgPath` other than it matching the equivalent | ||
// argument passed to `Apply()`, it is typically sourced from | ||
// `golang.org/x/tools/go/packages.Package.PkgPath`. | ||
func (r PatchRegistry) Add(pkgPath string, zeroNode ast.Node, fn Patch) { | ||
pkg, ok := r[pkgPath] | ||
if !ok { | ||
pkg = make(map[reflect.Type][]Patch) | ||
r[pkgPath] = pkg | ||
} | ||
|
||
t := nodeType(zeroNode) | ||
pkg[t] = append(pkg[t], fn) | ||
} | ||
|
||
// Apply calls `astutil.Apply()` on `node`, calling the appropriate `Patch` | ||
// functions as the syntax tree is traversed. Patches are applied as the `pre` | ||
// argument to `astutil.Apply()`. | ||
// | ||
// Global `pkgPath` matches (i.e. to those registered with "*") will be applied | ||
// before package-specific matches. | ||
// | ||
// If any `Patch` returns an error then no further patches will be called, and | ||
// the error will be returned by `Apply()`. | ||
func (r PatchRegistry) Apply(pkgPath string, node ast.Node) error { | ||
var err error | ||
astutil.Apply(node, func(c *astutil.Cursor) bool { | ||
if err != nil { | ||
return false | ||
} | ||
if err = r.applyToCursor("*", c); err != nil { | ||
return false | ||
} | ||
err = r.applyToCursor(pkgPath, c) | ||
return err == nil | ||
}, nil) | ||
return err | ||
} | ||
|
||
func (r PatchRegistry) applyToCursor(pkgPath string, c *astutil.Cursor) error { | ||
if c.Node() == nil { | ||
return nil | ||
} | ||
|
||
pkg, ok := r[pkgPath] | ||
if !ok { | ||
return nil | ||
} | ||
for _, fn := range pkg[nodeType(c.Node())] { | ||
if err := fn(c); err != nil { | ||
return err | ||
} | ||
} | ||
return nil | ||
} | ||
|
||
// nodeType returns the `reflect.Type` of the _concrete_ type implementing | ||
// `ast.Node`. Simpy calling `reflect.TypeOf(n)` would be incorrect as it would | ||
// reflect the interface (and not match any nodes). | ||
func nodeType(n ast.Node) reflect.Type { | ||
return reflect.ValueOf(n).Type() | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,121 @@ | ||
package astpatch | ||
|
||
import ( | ||
"bytes" | ||
"fmt" | ||
"go/ast" | ||
"go/parser" | ||
"go/token" | ||
"testing" | ||
|
||
"github.com/google/go-cmp/cmp" | ||
"github.com/stretchr/testify/assert" | ||
"github.com/stretchr/testify/require" | ||
"golang.org/x/tools/go/ast/astutil" | ||
) | ||
|
||
type patchSpy struct { | ||
gotFuncs, gotStructs []string | ||
} | ||
|
||
const errorIfFuncName = "ErrorFuncName" | ||
|
||
var errFuncName = fmt.Errorf("encountered sentinel function %q", errorIfFuncName) | ||
|
||
func (s *patchSpy) funcRecorder(c *astutil.Cursor) error { | ||
name := c.Node().(*ast.FuncDecl).Name.String() | ||
if name == errorIfFuncName { | ||
return errFuncName | ||
} | ||
s.gotFuncs = append(s.gotFuncs, name) | ||
return nil | ||
} | ||
|
||
func (s *patchSpy) structRecorder(c *astutil.Cursor) error { | ||
switch p := c.Parent().(type) { | ||
case *ast.TypeSpec: // it's a `type x struct` not, for example, a `map[T]struct{}` | ||
s.gotStructs = append(s.gotStructs, p.Name.String()) | ||
} | ||
return nil | ||
} | ||
|
||
func TestPatchRegistry(t *testing.T) { | ||
tests := []struct { | ||
name string | ||
src string | ||
wantErr error | ||
wantFuncs, wantStructs []string | ||
}{ | ||
{ | ||
name: "happy path", | ||
src: `package thepackage | ||
|
||
func FnA(){} | ||
|
||
func FnB(){} | ||
|
||
type StructA struct{} | ||
|
||
type StructB struct{} | ||
`, | ||
wantFuncs: []string{"FnA", "FnB"}, | ||
wantStructs: []string{"StructA", "StructB"}, | ||
}, | ||
{ | ||
name: "error propagation", | ||
src: `package thepackage | ||
|
||
func HappyFn() {} | ||
|
||
func ` + errorIfFuncName + `() {} | ||
`, | ||
wantErr: errFuncName, | ||
wantFuncs: []string{"HappyFn"}, | ||
}, | ||
} | ||
|
||
for _, tt := range tests { | ||
t.Run(tt.name, func(t *testing.T) { | ||
var spy patchSpy | ||
reg := make(PatchRegistry) | ||
|
||
reg.Add("*", &ast.FuncDecl{}, spy.funcRecorder) | ||
const pkgPath = `github.com/the/repo/thepackage` | ||
reg.Add(pkgPath, &ast.StructType{}, spy.structRecorder) | ||
|
||
reg.Add("unknown/package/path", &ast.FuncDecl{}, func(c *astutil.Cursor) error { | ||
t.Errorf("unexpected call to %T with different package path", (Patch)(nil)) | ||
return nil | ||
}) | ||
|
||
file := parseGoFile(t, token.NewFileSet(), tt.src) | ||
bestEffortLogAST(t, file) | ||
|
||
// None of the `require.Equal*()` variants provide a check for exact | ||
// match (i.e. equivalent to ==) of the identical error being | ||
// propagated. | ||
if gotErr := reg.Apply(pkgPath, file); gotErr != tt.wantErr { | ||
t.Fatalf("%T.Apply(...) got err %v; want %v", reg, gotErr, tt.wantErr) | ||
} | ||
assert.Empty(t, cmp.Diff(tt.wantFuncs, spy.gotFuncs), "encountered function declarations (-want +got)") | ||
assert.Empty(t, cmp.Diff(tt.wantStructs, spy.gotStructs), "encountered struct-type declarations (-want +got)") | ||
}) | ||
} | ||
} | ||
|
||
func parseGoFile(tb testing.TB, fset *token.FileSet, src string) *ast.File { | ||
tb.Helper() | ||
f, err := parser.ParseFile(fset, "", src, parser.SkipObjectResolution) | ||
require.NoError(tb, err, "Parsing Go source as file: parser.ParseFile([see logged source])") | ||
return f | ||
} | ||
|
||
func bestEffortLogAST(tb testing.TB, x any) { | ||
tb.Helper() | ||
|
||
var buf bytes.Buffer | ||
if err := ast.Fprint(&buf, nil, x, nil); err != nil { | ||
return | ||
} | ||
tb.Logf("AST of parsed source:\n\n%s", buf.String()) | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
// (c) 2019-2024, Ava Labs, Inc. | ||
// | ||
// This file is a derived work, based on the go-ethereum library whose original | ||
// notices appear below. | ||
// | ||
// It is distributed under a license compatible with the licensing terms of the | ||
// original code from which it is derived. | ||
// | ||
// Much love to the original authors for their work. | ||
// ********** | ||
// | ||
// Code generated by gethclone - DO NOT EDIT. | ||
// | ||
// ********** |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a false positive on
const geth = "github.com/ethereum/go-ethereum/"
so I updated the check to only match with (possible) whitespace but no other prefix.