Skip to content

Commit 99de31d

Browse files
committed
Switch to a linked-list stack
This makes other changes too: * uses an Int for the program counter * removes Julia from many names, since Julia is in the name of the package * improves the API consistency of higher-order commands
1 parent d36f8e3 commit 99de31d

19 files changed

+999
-857
lines changed

docs/src/dev_reference.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ JuliaInterpreter.get_return
3333
JuliaInterpreter.next_until!
3434
JuliaInterpreter.through_methoddef_or_done!
3535
JuliaInterpreter.evaluate_call!
36-
JuliaInterpreter.evaluate_foreigncall!
36+
JuliaInterpreter.evaluate_foreigncall
3737
JuliaInterpreter.maybe_evaluate_builtin
3838
```
3939

@@ -50,9 +50,9 @@ remove
5050
## Types
5151

5252
```@docs
53-
JuliaInterpreter.JuliaStackFrame
54-
JuliaInterpreter.JuliaFrameCode
55-
JuliaInterpreter.JuliaProgramCounter
53+
JuliaInterpreter.Frame
54+
JuliaInterpreter.FrameCode
55+
JuliaInterpreter.FrameData
5656
JuliaInterpreter.BreakpointState
5757
JuliaInterpreter.BreakpointRef
5858
```
@@ -69,7 +69,7 @@ JuliaInterpreter.compiled_methods
6969

7070
```@docs
7171
JuliaInterpreter.@lookup
72-
JuliaInterpreter.iswrappercall
72+
JuliaInterpreter.is_wrapper_call
7373
JuliaInterpreter.is_doc_expr
7474
JuliaInterpreter.is_global_ref
7575
JuliaInterpreter.statementnumber

docs/src/internals.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ let's build a frame:
99

1010
```julia
1111
julia> frame = JuliaInterpreter.enter_call(summer, A)
12-
JuliaStackFrame(JuliaInterpreter.JuliaFrameCode(summer(A::AbstractArray{T,N} where N) where T in Main at REPL[1]:2, CodeInfo(
12+
Frame(JuliaInterpreter.FrameCode(summer(A::AbstractArray{T,N} where N) where T in Main at REPL[1]:2, CodeInfo(
1313
1 ─ s = ($(QuoteNode(zero)))($(Expr(:static_parameter, 1)))
1414
%2 = A
1515
#temp# = ($(QuoteNode(iterate)))(%2)
@@ -29,8 +29,8 @@ JuliaStackFrame(JuliaInterpreter.JuliaFrameCode(summer(A::AbstractArray{T,N} whe
2929
), Core.TypeMapEntry[#undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef], BitSet([2, 4, 5, 7, 9, 12, 13]), false, false, true), Union{Nothing, Some{Any}}[Some(summer), Some([1, 2, 5]), nothing, nothing, nothing], Any[#undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef, #undef], Any[Int64], Int64[], Base.RefValue{Any}(nothing), Base.RefValue{JuliaInterpreter.JuliaProgramCounter}(JuliaProgramCounter(1)), Dict{Symbol,Int64}(), Any[])
3030
```
3131
32-
This is a [`JuliaInterpreter.JuliaStackFrame`](@ref). The `CodeInfo` is the most prominent part of this display,
33-
and extractable as `code = frame.code.code`. (It's a slightly modified form of one returned by `@code_lowered`,
32+
This is a [`JuliaInterpreter.Frame`](@ref). The `CodeInfo` is the most prominent part of this display,
33+
and extractable as `code = frame.framecode.code`. (It's a slightly modified form of one returned by `@code_lowered`,
3434
in that it has been processed by [`JuliaInterpreter.optimize!`](@ref) to speed up run-time execution.)
3535
3636
Much of the rest of the `frame` holds values needed for or generated by execution.
@@ -49,10 +49,10 @@ julia> frame.locals
4949
These correspond to the `code.slotnames`; the first is the `#self#` argument and the second
5050
is the input array. The remaining local variables (e.g., `s` and `a`), have not yet been assigned---we've
5151
only built the frame, but we haven't yet begun to execute it.
52-
The static parameter, `T`, is stored in `frame.sparams`:
52+
The static parameter, `T`, is stored in `frame.framedata.sparams`:
5353
5454
```julia
55-
julia> frame.sparams
55+
julia> frame.framedata.sparams
5656
1-element Array{Any,1}:
5757
Int64
5858
```
@@ -88,7 +88,7 @@ The other main entity is the so-called [program counter](https://en.wikipedia.or
8888
which just indicates the next statement to be executed:
8989
9090
```julia
91-
julia> frame.pc[]
91+
julia> frame.pc
9292
JuliaProgramCounter(1)
9393
```
9494
@@ -98,8 +98,8 @@ Let's try executing the first statement. So that we can recurse into calls (e.g.
9898
we'll create a [stack](https://en.wikipedia.org/wiki/Call_stack) of frames and then run the first statement:
9999
100100
```julia
101-
julia> stack = JuliaInterpreter.JuliaStackFrame[]
102-
0-element Array{JuliaStackFrame,1}
101+
julia> stack = JuliaInterpreter.Frame[]
102+
0-element Array{Frame,1}
103103

104104
julia> JuliaInterpreter.step_expr!(stack, frame)
105105
JuliaProgramCounter(2)
@@ -171,7 +171,7 @@ ex = quote
171171
end
172172

173173
frame = JuliaInterpreter.prepare_thunk(Main, ex)
174-
JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame)
174+
JuliaInterpreter.finish_and_return!(Frame[], frame)
175175

176176
# output
177177

@@ -196,7 +196,7 @@ Here's a demonstration of the problem:
196196
```julia
197197
ex = :(map(x->x^2, [1, 2, 3]))
198198
frame = JuliaInterpreter.prepare_thunk(Main, ex)
199-
julia> JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame)
199+
julia> JuliaInterpreter.finish_and_return!(Frame[], frame)
200200
ERROR: this frame needs to be run a top level
201201
```
202202
@@ -235,7 +235,7 @@ function" for this type, equivalent to `(##17#18)(x) = x^2`.
235235
In some cases one can fix this simply by indicating that we want to run this frame at top level:
236236
237237
```julia
238-
julia> JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame, true)
238+
julia> JuliaInterpreter.finish_and_return!(Frame[], frame, true)
239239
3-element Array{Int64,1}:
240240
1
241241
4
@@ -247,7 +247,7 @@ for more complex situations where there may be nested calls of new methods):
247247
248248
```julia
249249
modexs, _ = JuliaInterpreter.split_expressions(Main, ex)
250-
stack = JuliaStackFrame[]
250+
stack = Frame[]
251251
for (mod, e) in modexs
252252
frame = JuliaInterpreter.prepare_thunk(mod, e)
253253
while true

src/JuliaInterpreter.jl

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ using Random.DSFMT
1212
using InteractiveUtils
1313
using CodeTracking
1414

15-
export @interpret, Compiled, JuliaStackFrame,
16-
Breakpoints, breakpoint, @breakpoint, breakpoints, enable, disable, remove
15+
export @interpret, Compiled, Frame, root, leaf,
16+
BreakpointRef, breakpoint, @breakpoint, breakpoints, enable, disable, remove
1717

1818
module CompiledCalls
1919
# This module is for handling intrinsics that must be compiled (llvmcall)
@@ -26,6 +26,7 @@ include("localmethtable.jl")
2626
include("interpret.jl")
2727
include("builtins-julia$(Int(VERSION.major)).$(Int(VERSION.minor)).jl")
2828
include("optimize.jl")
29+
include("commands.jl")
2930
include("breakpoints.jl")
3031

3132
function set_compiled_methods()

src/breakpoints.jl

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -21,37 +21,38 @@ function add_breakpoint(framecode, stmtidx)
2121
return bp
2222
end
2323

24-
function shouldbreak(frame, pc=frame.pc[])
25-
idx = convert(Int, pc)
26-
isassigned(frame.code.breakpoints, idx) || return false
27-
bp = frame.code.breakpoints[idx]
24+
function shouldbreak(frame, pc=frame.pc)
25+
idx = pc
26+
bps = frame.framecode.breakpoints
27+
isassigned(bps, idx) || return false
28+
bp = bps[idx]
2829
bp.isactive || return false
2930
return Base.invokelatest(bp.condition, frame)::Bool
3031
end
3132

32-
function prepare_slotfunction(framecode::JuliaFrameCode, body::Union{Symbol,Expr})
33+
function prepare_slotfunction(framecode::FrameCode, body::Union{Symbol,Expr})
3334
ismeth = framecode.scope isa Method
3435
uslotnames = Set{Symbol}()
3536
slotnames = Symbol[]
36-
for name in framecode.code.slotnames
37+
for name in framecode.src.slotnames
3738
if name uslotnames
3839
push!(slotnames, name)
3940
push!(uslotnames, name)
4041
end
4142
end
42-
assignments = Expr[]
43-
framename = gensym("frame")
43+
framename, dataname = gensym("frame"), gensym("data")
44+
assignments = Expr[:($dataname = $framename.framedata)]
4445
default = Unassigned()
4546
for i = 1:length(slotnames)
46-
slotname = framecode.code.slotnames[i]
47+
slotname = framecode.src.slotnames[i]
4748
qslotname = QuoteNode(slotname)
48-
getexpr = :(something($framename.locals[$framename.last_reference[$qslotname]]))
49-
push!(assignments, Expr(:(=), slotname, :(haskey($framename.last_reference, $qslotname) ? $getexpr : $default)))
49+
getexpr = :(something($dataname.locals[$dataname.last_reference[$qslotname]]))
50+
push!(assignments, Expr(:(=), slotname, :(haskey($dataname.last_reference, $qslotname) ? $getexpr : $default)))
5051
end
5152
if ismeth
5253
syms = sparam_syms(framecode.scope)
5354
for i = 1:length(syms)
54-
push!(assignments, Expr(:(=), syms[i], :($framename.sparams[$i])))
55+
push!(assignments, Expr(:(=), syms[i], :($dataname.sparams[$i])))
5556
end
5657
end
5758
funcname = ismeth ? gensym("slotfunction") : gensym(Symbol(framecode.scope.name, "_slotfunction"))
@@ -62,8 +63,8 @@ const Condition = Union{Nothing,Expr,Tuple{Module,Expr}}
6263
_unpack(condition) = isa(condition, Expr) ? (Main, condition) : condition
6364

6465
## The fundamental implementations of breakpoint-setting
65-
function breakpoint!(framecode::JuliaFrameCode, pc, condition::Condition=nothing)
66-
stmtidx = convert(Int, pc)
66+
function breakpoint!(framecode::FrameCode, pc, condition::Condition=nothing)
67+
stmtidx = pc
6768
if condition === nothing
6869
framecode.breakpoints[stmtidx] = BreakpointState()
6970
else
@@ -73,8 +74,8 @@ function breakpoint!(framecode::JuliaFrameCode, pc, condition::Condition=nothing
7374
end
7475
return add_breakpoint(framecode, stmtidx)
7576
end
76-
breakpoint!(frame::JuliaStackFrame, pc=frame.pc[], condition::Condition=nothing) =
77-
breakpoint!(frame.code, pc, condition)
77+
breakpoint!(frame::Frame, pc=frame.pc, condition::Condition=nothing) =
78+
breakpoint!(frame.framecode, pc, condition)
7879

7980
"""
8081
enable(bp::BreakpointRef)

src/commands.jl

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
"""
2+
ret = finish!(recurse, frame, istoplevel=false)
3+
ret = finish!(frame, istoplevel=false)
4+
5+
Run `frame` until execution terminates. `ret` is either `nothing` (if execution terminates
6+
when it hits a `return` statement) or a reference to a breakpoint.
7+
In the latter case, `leaf(frame)` returns the frame in which it hit the breakpoint.
8+
9+
`recurse` controls call evaluation; `recurse = Compiled()` evaluates :call expressions
10+
by normal dispatch, whereas the default `recurse = finish_and_return!` uses recursive interpretation.
11+
"""
12+
function finish!(@nospecialize(recurse), frame::Frame, istoplevel::Bool=false)
13+
local pc
14+
while true
15+
pc = step_expr!(recurse, frame, istoplevel)
16+
(pc == nothing || isa(pc, BreakpointRef)) && return pc
17+
shouldbreak(frame, pc) && return BreakpointRef(frame.framecode, pc)
18+
end
19+
end
20+
finish!(frame::Frame, istoplevel::Bool=false) = finish!(finish_and_return!, frame, istoplevel)
21+
22+
"""
23+
ret = finish_and_return!(recurse, frame, istoplevel::Bool=false)
24+
ret = finish_and_return!(frame, istoplevel::Bool=false)
25+
26+
Call [`JuliaInterpreter.finish!`](@ref) and pass back the return value. If execution
27+
pauses at a breakpoint, the reference to the breakpoint is returned.
28+
"""
29+
function finish_and_return!(@nospecialize(recurse), frame::Frame, istoplevel::Bool=false)
30+
pc = finish!(recurse, frame, istoplevel)
31+
isa(pc, BreakpointRef) && return pc
32+
return get_return(frame)
33+
end
34+
finish_and_return!(frame::Frame, istoplevel::Bool=false) = finish_and_return!(finish_and_return!, frame, istoplevel)
35+
36+
"""
37+
bpref = dummy_breakpoint(recurse, frame::Frame)
38+
39+
Return a fake breakpoint. This can be useful as the `recurse` argument to `evaluate_call!`
40+
(or any of the higher-order commands) to ensure that you return immediately after stepping
41+
into a call.
42+
"""
43+
dummy_breakpoint(@nospecialize(recurse), frame::Frame) = BreakpointRef(frame.framecode, 0)
44+
45+
"""
46+
ret = finish_stack!(recurse, frame, istoplevel=false)
47+
ret = finish_stack!(frame, istoplevel=false)
48+
49+
Unwind the callees of `frame`, finishing each before returning to the caller.
50+
`frame` itself is also finished
51+
If execution hits a breakpoint, `ret` will be a reference to the breakpoint.
52+
"""
53+
function finish_stack!(@nospecialize(recurse), frame::Frame, istoplevel::Bool=false)
54+
frame0 = frame
55+
frame = leaf(frame)
56+
while true
57+
ret = finish_and_return!(recurse, frame, istoplevel)
58+
isa(ret, BreakpointRef) && return ret
59+
frame === frame0 && return ret
60+
recycle(frame)
61+
frame = caller(frame)
62+
frame === nothing && return ret
63+
pc = frame.pc
64+
if isassign(frame, pc)
65+
lhs = getlhs(pc)
66+
do_assignment!(frame, lhs, ret)
67+
end
68+
pc += 1
69+
frame.pc = pc
70+
shouldbreak(frame) && return BreakpointRef(frame.framecode, pc)
71+
end
72+
return frame
73+
end
74+
finish_stack!(frame::Frame, istoplevel::Bool=false) = finish_stack!(finish_and_return!, frame, istoplevel)
75+
76+
"""
77+
next_until!(predicate, recurse, frame, istoplevel=false)
78+
next_until!(predicate, frame, istoplevel=false)
79+
80+
Execute the current statement. Then step through statements of `frame` until the next
81+
statement satifies `predicate(stmt)`.
82+
"""
83+
function next_until!(@nospecialize(predicate), @nospecialize(recurse), frame::Frame, istoplevel::Bool=false)
84+
pc = step_expr!(recurse, frame, istoplevel)
85+
while pc !== nothing && !isa(pc, BreakpointRef)
86+
if predicate(pc_expr(frame, pc)) || shouldbreak(frame, pc)
87+
return pc
88+
end
89+
pc = step_expr!(recurse, frame, istoplevel)
90+
end
91+
return pc
92+
end
93+
next_until!(predicate, frame::Frame, istoplevel::Bool=false) =
94+
next_until!(predicate, finish_and_return!, frame, istoplevel)
95+
96+
next_call!(@nospecialize(recurse), frame::Frame, istoplevel::Bool=false) =
97+
next_until!(is_call_or_return, recurse, frame, istoplevel)
98+
next_call!(frame::Frame, istoplevel::Bool=false) = next_call!(finish_and_return!, frame, istoplevel)
99+
100+
function maybe_next_call!(@nospecialize(recurse), frame::Frame, istoplevel::Bool=false)
101+
pc = frame.pc
102+
is_call_or_return(pc_expr(frame, pc)) && return pc
103+
return next_call!(recurse, frame, istoplevel)
104+
end
105+
maybe_next_call!(frame::Frame, istoplevel::Bool=false) =
106+
maybe_next_call!(finish_and_return!, frame, istoplevel)
107+
108+
"""
109+
through_methoddef_or_done!(recurse, frame)
110+
through_methoddef_or_done!(frame)
111+
112+
Runs `frame` at top level until it either finishes (e.g., hits a `return` statement)
113+
or defines a new method.
114+
"""
115+
function through_methoddef_or_done!(@nospecialize(recurse), frame::Frame)
116+
predicate(stmt) = isexpr(stmt, :method, 3) || isexpr(stmt, :thunk)
117+
pc = next_until!(predicate, recurse, frame, true)
118+
(pc === nothing || isa(pc, BreakpointRef)) && return pc
119+
return step_expr!(recurse, frame, true) # define the method and return
120+
end
121+
through_methoddef_or_done!(@nospecialize(recurse), t::Tuple{Module,Expr,Frame}) =
122+
through_methoddef_or_done!(recurse, t[end])
123+
through_methoddef_or_done!(@nospecialize(recurse), modex::Tuple{Module,Expr,Expr}) = Core.eval(modex[1], modex[3])
124+
through_methoddef_or_done!(arg) = through_methoddef_or_done!(finish_and_return!, arg)
125+
126+
function changed_line!(expr, line, fls)
127+
if length(fls) == 1 && isa(expr, LineNumberNode)
128+
return expr.line != line
129+
elseif length(fls) == 1 && isa(expr, Expr) && isexpr(expr, :line)
130+
return expr.args[1] != line
131+
else
132+
if is_loc_meta(expr, :pop_loc)
133+
pop!(fls)
134+
elseif is_loc_meta(expr, :push_loc)
135+
push!(fls,(expr.args[2],0))
136+
end
137+
return false
138+
end
139+
end
140+
141+
function next_line!(@nospecialize(recurse), frame::Frame, istoplevel::Bool=false)
142+
initial = linenumber(frame)
143+
first = true
144+
pc = frame.pc
145+
while linenumber(frame, pc) == initial
146+
# If this is a return node, interrupt execution
147+
expr = pc_expr(frame, pc)
148+
(!first && isexpr(expr, :return)) && return pc
149+
first = false
150+
# If this is a goto node, step it and reevaluate
151+
if is_goto_node(expr)
152+
pc = step_expr!(recurse, frame, istoplevel)
153+
(pc === nothing || isa(pc, BreakpointRef)) && return pc
154+
elseif recurse !== nothing && is_wrapper_call(expr)
155+
# With splatting it can happen that we do something like ssa = tuple(#self#), _apply(ssa), which
156+
# confuses the logic here, just step into the first call that's not a builtin
157+
switched = false
158+
while is_wrapper_call(expr)
159+
ret = evaluate_call!(dummy_breakpoint, frame, expr)
160+
if frame.callee === nothing && !isa(ret, BreakpointRef)
161+
# This wasn't a real wrapper call
162+
if isassign(frame, pc)
163+
lhs = getlhs(pc)
164+
do_assignment!(frame, lhs, ret)
165+
end
166+
frame.pc = pc = pc + 1
167+
break
168+
end
169+
frame = frame.callee
170+
switched = true
171+
expr = pc_expr(frame)
172+
end
173+
# Signal that we've switched frames
174+
switched && return BreakpointRef(frame.framecode, frame.pc)
175+
else
176+
pc = step_expr!(recurse, frame, istoplevel)
177+
(pc === nothing || isa(pc, BreakpointRef)) && return pc
178+
end
179+
shouldbreak(frame, pc) && return BreakpointRef(frame.framecode, pc)
180+
end
181+
maybe_next_call!(recurse, frame, pc)
182+
end
183+
next_line!(frame::Frame, istoplevel::Bool=false) = next_line!(finish_and_return!, frame, istoplevel)

0 commit comments

Comments
 (0)