|
| 1 | +# MacroTools.jl |
| 2 | + |
| 3 | +A library provides helpful tools for writing macros, notably a very simple but |
| 4 | +powerful templating system and some functions that have proven useful to me. |
| 5 | + |
| 6 | +## Template Matching |
| 7 | + |
| 8 | +Template matching enables macro writers to deconstruct Julia |
| 9 | +expressions in a more declarative way, and without having to know in |
| 10 | +great detail how syntax is represented internally. For example, say you |
| 11 | +have a type definition: |
| 12 | + |
| 13 | +```julia |
| 14 | +ex = quote |
| 15 | + struct Foo |
| 16 | + x::Int |
| 17 | + y |
| 18 | + end |
| 19 | +end |
| 20 | +``` |
| 21 | + |
| 22 | +If you know what you're doing, you can pull out the name and fields via: |
| 23 | + |
| 24 | +```julia |
| 25 | +julia> if isexpr(ex.args[2], :struct) |
| 26 | + (ex.args[2].args[2], ex.args[2].args[3].args) |
| 27 | + end |
| 28 | +(:Foo,{:( # line 3:),:(x::Int),:( # line 4:),:y}) |
| 29 | +``` |
| 30 | +
|
| 31 | +But this is hard to write – since you have to deconstruct the `type` |
| 32 | +expression by hand – and hard to read, since you can't tell at a glance |
| 33 | +what's being achieved. On top of that, there's a bunch of messy stuff to |
| 34 | +deal with like pesky `begin` blocks which wrap a single expression, line |
| 35 | +numbers, etc. etc. |
| 36 | +
|
| 37 | +Enter MacroTools: |
| 38 | +
|
| 39 | +```julia |
| 40 | +julia> using MacroTools |
| 41 | + |
| 42 | +julia> @capture(ex, struct T_ fields__ end) |
| 43 | +true |
| 44 | + |
| 45 | +julia> T, fields |
| 46 | +(:Foo, [:(x::Int), :y]) |
| 47 | +``` |
| 48 | +
|
| 49 | +Symbols like `T_` underscore are treated as catchalls which match any |
| 50 | +expression, and the expression they match is bound to the |
| 51 | +(underscore-less) variable, as above. |
| 52 | +
|
| 53 | +Because `@capture` doubles as a test as well as extracting values, you can |
| 54 | +easily handle unexpected input (try writing this by hand): |
| 55 | +
|
| 56 | +```julia |
| 57 | +@capture(ex, f_{T_}(xs__) = body_) || |
| 58 | + error("expected a function with a single type parameter") |
| 59 | +``` |
| 60 | +
|
| 61 | +Symbols like `f__` (double underscored) are similar, but slurp a sequence of |
| 62 | +arguments into an array. For example: |
| 63 | +
|
| 64 | +```julia |
| 65 | +julia> @capture(:[1, 2, 3, 4, 5, 6, 7], [1, a_, 3, b__, c_]) |
| 66 | +true |
| 67 | + |
| 68 | +julia> a, b, c |
| 69 | +(2,[4,5,6],7) |
| 70 | +``` |
| 71 | +
|
| 72 | +Slurps don't have to be at the end of an expression, but like the |
| 73 | +Highlander there can only be one (per expression). |
| 74 | +
|
| 75 | +### Matching on expression type |
| 76 | +
|
| 77 | +`@capture` can match expressions by their type, which is either the `head` of `Expr` |
| 78 | +objects or the `typeof` atomic stuff like `Symbol`s and `Int`s. For example: |
| 79 | +
|
| 80 | +```julia |
| 81 | +@capture(ex, foo(x_String_string)) |
| 82 | +``` |
| 83 | +
|
| 84 | +This will match a call to the `foo` function which has a single argument, which |
| 85 | +may either be a `String` object or a `Expr(:string, ...)` |
| 86 | +(e.g. `@capture(:(foo("$(a)")), foo(x_String_string))`). Julia string literals |
| 87 | +may be parsed into either type of object, so this is a handy way to catch both. |
| 88 | +
|
| 89 | +Another common use case is to catch symbol literals, e.g. |
| 90 | +
|
| 91 | +```julia |
| 92 | +@capture(ex, |
| 93 | + struct T_Symbol |
| 94 | + fields__ |
| 95 | + end) |
| 96 | +``` |
| 97 | +
|
| 98 | +which will match e.g. `struct Foo ...` but not `struct Foo{V} ...` |
| 99 | +
|
| 100 | +### Unions |
| 101 | +
|
| 102 | +`@capture` can also try to match the expression against one pattern or another, |
| 103 | +for example: |
| 104 | +
|
| 105 | +```julia |
| 106 | +@capture(ex, f_(args__) = body_ | function f_(args__) body_ end) |
| 107 | +``` |
| 108 | +
|
| 109 | +will match both kinds of function syntax (though it's easier to use |
| 110 | +`shortdef` to normalise definitions). You can also do this within |
| 111 | +expressions, e.g. |
| 112 | +
|
| 113 | +```julia |
| 114 | +@capture(ex, (f_{T_}|f_)(args__) = body_) |
| 115 | +``` |
| 116 | +
|
| 117 | +matches a function definition, with a single type parameter bound to `T` if possible. |
| 118 | +If not, `T = nothing`. |
| 119 | +
|
| 120 | +## Expression Walking |
| 121 | +
|
| 122 | +If you've ever written any more interesting macros, you've probably found |
| 123 | +yourself writing recursive functions to work with nested `Expr` trees. |
| 124 | +MacroTools' `prewalk` and `postwalk` functions factor out the recursion, making |
| 125 | +macro code much more concise and robust. |
| 126 | +
|
| 127 | +These expression-walking functions essentially provide a kind of |
| 128 | +find-and-replace for expression trees. For example: |
| 129 | +
|
| 130 | +```julia |
| 131 | +julia> using MacroTools: prewalk, postwalk |
| 132 | + |
| 133 | +julia> postwalk(x -> x isa Integer ? x + 1 : x, :(2+3)) |
| 134 | +:(3 + 4) |
| 135 | +``` |
| 136 | +
|
| 137 | +In other words, look at each item in the tree; if it's an integer, add one, if not, leave it alone. |
| 138 | +
|
| 139 | +We can do more complex things if we combine this with `@capture`. For example, say we want to insert an extra argument into all function calls: |
| 140 | +
|
| 141 | +```julia |
| 142 | +julia> ex = quote |
| 143 | + x = f(y, g(z)) |
| 144 | + return h(x) |
| 145 | + end |
| 146 | + |
| 147 | +julia> postwalk(x -> @capture(x, f_(xs__)) ? :($f(5, $(xs...))) : x, ex) |
| 148 | +quote # REPL[20], line 2: |
| 149 | + x = f(5, y, g(5, z)) # REPL[20], line 3: |
| 150 | + return h(5, x) |
| 151 | +end |
| 152 | +``` |
| 153 | +
|
| 154 | +Most of the time, you can use `postwalk` without worrying about it, but we also |
| 155 | +provide `prewalk`. The difference is the order in which you see sub-expressions; |
| 156 | +`postwalk` sees the leaves of the `Expr` tree first and the whole expression |
| 157 | +last, while `prewalk` is the opposite. |
| 158 | +
|
| 159 | +```julia |
| 160 | +julia> postwalk(x -> @show(x) isa Integer ? x + 1 : x, :(2+3*4)); |
| 161 | +x = :+ |
| 162 | +x = 2 |
| 163 | +x = :* |
| 164 | +x = 3 |
| 165 | +x = 4 |
| 166 | +x = :(4 * 5) |
| 167 | +x = :(3 + 4 * 5) |
| 168 | + |
| 169 | +julia> prewalk(x -> @show(x) isa Integer ? x + 1 : x, :(2+3*4)); |
| 170 | +x = :(2 + 3 * 4) |
| 171 | +x = :+ |
| 172 | +x = 2 |
| 173 | +x = :(3 * 4) |
| 174 | +x = :* |
| 175 | +x = 3 |
| 176 | +x = 4 |
| 177 | +``` |
| 178 | +
|
| 179 | +A significant difference is that `prewalk` will walk into whatever expression |
| 180 | +you return. |
| 181 | +
|
| 182 | +```julia |
| 183 | +julia> postwalk(x -> @show(x) isa Integer ? :(a+b) : x, 2) |
| 184 | +x = 2 |
| 185 | +:(a + b) |
| 186 | + |
| 187 | +julia> prewalk(x -> @show(x) isa Integer ? :(a+b) : x, 2) |
| 188 | +x = 2 |
| 189 | +x = :+ |
| 190 | +x = :a |
| 191 | +x = :b |
| 192 | +:(a + b) |
| 193 | +``` |
| 194 | +
|
| 195 | +This makes it somewhat more prone to infinite loops; for example, if we returned |
| 196 | +`:(1+b)` instead of `:(a+b)`, `prewalk` would hang trying to expand all of the |
| 197 | +`1`s in the expression. |
| 198 | +
|
| 199 | +With these tools in hand, a useful general pattern for macros is: |
| 200 | +
|
| 201 | +```julia |
| 202 | +macro foo(ex) |
| 203 | + postwalk(ex) do x |
| 204 | + @capture(x, some_pattern) || return x |
| 205 | + return new_x |
| 206 | + end |
| 207 | +end |
| 208 | +``` |
| 209 | +
|
| 210 | +## Function definitions |
| 211 | +
|
| 212 | +`splitdef(def)` matches a function definition of the form |
| 213 | +
|
| 214 | +```julia |
| 215 | +function name{params}(args; kwargs)::rtype where {whereparams} |
| 216 | + body |
| 217 | +end |
| 218 | +``` |
| 219 | +
|
| 220 | +and returns `Dict(:name=>..., :args=>..., etc.)`. The definition can be rebuilt by |
| 221 | +calling `MacroTools.combinedef(dict)`, or explicitly with |
| 222 | +
|
| 223 | +```julia |
| 224 | +rtype = get(dict, :rtype, :Any) |
| 225 | +all_params = [get(dict, :params, [])..., get(dict, :whereparams, [])...] |
| 226 | +:(function $(dict[:name]){$(all_params...)}($(dict[:args]...); |
| 227 | + $(dict[:kwargs]...))::$rtype |
| 228 | + $(dict[:body]) |
| 229 | + end) |
| 230 | +``` |
| 231 | +
|
| 232 | +`splitarg(arg)` matches function arguments (whether from a definition or a function call) |
| 233 | +such as `x::Int=2` and returns `(arg_name, arg_type, slurp, default)`. `default` is |
| 234 | +`nothing` when there is none. For example: |
| 235 | +
|
| 236 | +```julia |
| 237 | +> map(splitarg, (:(f(y, a=2, x::Int=nothing, args...))).args[2:end]) |
| 238 | +4-element Array{Tuple{Symbol,Symbol,Bool,Any},1}: |
| 239 | + (:y, :Any, false, nothing) |
| 240 | + (:a, :Any, false, 2) |
| 241 | + (:x, :Int, false, :nothing) |
| 242 | + (:args, :Any, true, nothing) |
| 243 | +``` |
| 244 | +
|
| 245 | +## Function and Macros |
| 246 | +
|
| 247 | +```@autodocs |
| 248 | +Modules = [MacroTools] |
| 249 | +Order = [:module, :constant, :type, :macro, :function] |
| 250 | +``` |
0 commit comments