Running things for real: a roadmap

[sampsyo]

As a recreational project this winter, I poked around at our infrastructure for running programs for real on Xilinx FPGAs (which is all the incredible work of the inimitable @sgpthomas!!). I just wanted to tie together the issues I've been filing to summarize the current state of things, which might be especially relevant to @yn224.

The bottom line is: compilation is working OK, with one significant asterisk; emulation is barely starting to work; and I have not tried real FPGA execution.

• Compilation: As of [fud] Make SSH optional in Xilinx synthesis (xclbin) stage #850, [fud] Add save_temps option to Xilinx synthesis #851, [fud] Make SSH optional in Xilinx emulation stage #852, and [fud] Simplify & document .xo production #855, we are successfully producing xclbin files from Calyx programs. 🎉
  • The big remaining problem is an issue involving multiple memories and multiple AXI interfaces, described in [xilinx] Use a single AXI interface to communicate with all memories #853.
  • The next step to be done here is to see if the xo generation Tcl needs different declarations of the AXI interfaces, as described in [xilinx] Use a single AXI interface to communicate with all memories #853 (comment).
  • It would also be useful to continue trying to simplify our Tcl script to the absolute bare minimum necessary to produce an xo file (so we understand exactly what we're doing).
• Emulation: You can see emulation working in the janky "tests" I made in [fud] Start some very tiny tests for Xilinx toolchain #866.
  • However, the one tiny test I am running is not producing the right answer. (The memory state seems to be unmodified from the initial state.) One next step is to debug this stuff.
  • There is also some important refactoring to be done in [fud] Revamp Xilinx fpga stage to work for both emulation and execution #872.
  • We are also missing docs for the fpga stage, which we should write after the refactoring. (We can delete the docs for the emulation stage.)
• Execution: Basically, we need to try out real execution on actual hardware. It only really makes sense to focus on it after emulation works, but we could do some of the experimentation concurrently. This would also benefit from the refactoring in the aforementioned [fud] Revamp Xilinx fpga stage to work for both emulation and execution #872.

Of course, the end result of all this should be that we can do fud e something.fuse --to dat --through fpga and everything just works (and the output matches our interpreter and Verilator execution). I also strongly believe we should maintain a focus on documenting things as thoroughly as we can possibly muster in the appropriate chapter—this stuff is so damned confusing and under-documented that we really benefit from writing things down clearly and exhaustively along the way.

Some fun future work after everything's nailed down for an MVP:

• Let's do Intel!! I keep hearing good things about OPAE, which is the Intel equivalent of Xilinx's XRT.
• Maybe we can remove our bespoke statistics-only Vivado synthesis setup and replace it with this toolchain. Who knows.

[rachitnigam]

Hey @sampsyo, what do you think about making this discussion into an issue instead? For good reasons or bad, I have a "discussion blindness" where I don't check discussions as often as issues to keep them updated and this looks like a really actionable, tracker-style issue. GitHub has a convenient "create issue from this discussion" feature.

[sampsyo]

Good call; done!