|
| 1 | +# SourceWalk |
| 2 | + |
| 3 | +Pattern matching is really useful if you're writing macros, but it also works |
| 4 | +on Julia source code files. You can try this with the `sourcewalk` function, |
| 5 | +which behaves like `postwalk` but with a source file as input. |
| 6 | + |
| 7 | +!!! note |
| 8 | + |
| 9 | + To use this functionality you currently need to a fork of CSTParser. |
| 10 | + ```julia |
| 11 | + ] add https://github.com/MikeInnes/CSTParser.jl#location |
| 12 | + ``` |
| 13 | + |
| 14 | +For example, if we have a file like |
| 15 | + |
| 16 | +```julia |
| 17 | +# test.jl |
| 18 | +foo(a, b) |
| 19 | +``` |
| 20 | + |
| 21 | +And run |
| 22 | + |
| 23 | +```julia |
| 24 | +sourcewalk("test.jl") do x |
| 25 | + x == :foo ? :bar : x |
| 26 | +end |
| 27 | +``` |
| 28 | + |
| 29 | +then our file will have changed to |
| 30 | + |
| 31 | +```julia |
| 32 | +# test.jl |
| 33 | +foo(a, b) |
| 34 | +``` |
| 35 | + |
| 36 | +You can also pass `sourcewalk` a directory, and it will work on all `.jl` files |
| 37 | +in that directory. You can use `sourcewalk` anywhere you'd normally use a |
| 38 | +Regex-based find and replace, but it'll take care of subtle issues like matching |
| 39 | +brackets, operator precedence and code formatting. |
| 40 | + |
| 41 | +We can use this to do some very powerful code transformations with minimal |
| 42 | +effort. For example, [this whole |
| 43 | +patch](https://github.com/MikeInnes/julia/commit/45ccbc6a3c003accb0eedca889835071c371ae86) |
| 44 | +was generated simply with `sourcewalk(longdef, "base")`. Let's look at some |
| 45 | +examples. |
| 46 | + |
| 47 | +!!! warning |
| 48 | + |
| 49 | + We recommend running SourceWalk on files that are checked into Git. This way |
| 50 | + you can easily look through every change and check it's sensible. Being |
| 51 | + careless with `sourcewalk` can delete all your code! |
| 52 | + |
| 53 | +## Find and Replace |
| 54 | + |
| 55 | +Here's a more realistic example. When working with strings, we use `nextind(s, i)` |
| 56 | +to get the next index after `i` – which will often just be `i+1`, but may |
| 57 | +be something else if we're working with unicode. |
| 58 | + |
| 59 | +How do you know your code handles unicode correctly? One way to check is to |
| 60 | +actually replace `nextind(s, i)` with `i + 1` and _make sure that the tests |
| 61 | +fail_ – if they don't, we're not testing with unicode input properly. This is a |
| 62 | +form of [mutation testing](https://en.wikipedia.org/wiki/Mutation_testing). |
| 63 | + |
| 64 | +Of course, the transformation is very simple with MacroTools: |
| 65 | + |
| 66 | +```julia |
| 67 | +julia> function nextinds(x) |
| 68 | + @capture(x, nextind(str_, i_)) && return :($i + 1) |
| 69 | + @capture(x, prevind(str_, i_)) && return :($i - 1) |
| 70 | + return x |
| 71 | + end |
| 72 | +nextinds (generic function with 1 method) |
| 73 | + |
| 74 | +julia> nextinds(:(nextind(a, b))) |
| 75 | +:(b + 1) |
| 76 | +``` |
| 77 | + |
| 78 | +We can check this works on text, before running it on our file, using `textwalk`. |
| 79 | + |
| 80 | +```julia |
| 81 | +julia> using MacroTools: textwalk |
| 82 | + |
| 83 | +julia> textwalk(nextinds, "nextind(s, i)") |
| 84 | +"i + 1" |
| 85 | +``` |
| 86 | + |
| 87 | +And verify that it correctly handles operator precedence – so we won't get invalid |
| 88 | +syntax or unexpected changes to the meaning of our code. |
| 89 | + |
| 90 | +```julia |
| 91 | +julia> textwalk(nextinds, "1:nextind(s, i)") |
| 92 | +"1:i + 1" |
| 93 | + |
| 94 | +julia> textwalk(nextinds, "1*nextind(s, i)") |
| 95 | +"1*(i + 1)" |
| 96 | +``` |
| 97 | + |
| 98 | +Running this on Julia's base code yields [the following patch](https://github.com/MikeInnes/julia/commit/b3964317321150c4b9ae8d629f613ee1143b3629). |
0 commit comments