Matrix Multiplication Cascade Programming Example

[kurtis-b]

In the MLIR-AIE dialect, the aie.cascade_flow sets the cascade connection between tiles. However, in the cascade programming example, it doesn't seem to be setting the cascade flow, so it's unclear how the cascade connection is being set--is it implicit somehow?

When I build the application, the generated MLIR also doesn't show the use of the cascade_flow operator. Moreover, in the kernel code, data is being written to the cascade stream, but there's also a third argument in the function call that doesn't seem to be used. Is that third argument contributing at all to the functionality of the application, and how is the compiler able to know which direction the cascade stream is supposed to flow (i.e. North to South, or West to East) without the cascade_flow operator?

[jackl-xilinx]

@kurtis-b It would be helpful if you link in the specific example you're referring to so if there's any errors, we can fix it.

• Cascade connections can be implicit as the physical connections for cascade are always present and enabled in hardware. By default, cascade directions should flow downwards. There is a selector that determines whether cascade outputs go south or east and cascade inputs from north or west. cascade flows are mainly used to make connections consistent and to configure these selectors.
• MLIR might not show the cascade flow operator but should have a placeholder for configuring the cascade selectors if it's already lowered from the cascade flow. Please share the design you have that's not doing this so we can look more closely at the issue.
• By kernel code code, which one are you referring to? the cascade instrinsics like get_scd_v16in32() and put_mcd which have 0 and 1 argument respectively. Other version for put_mcd has 2 arguments, 1 for enable. I'm not sure which function call you're referring to that has 3 args.

[kurtis-b]

Thanks for the response! That's my bad, I should've been more specific--the cascade programming example I'm referring to is the one in programming_examples/basic/matrix_multiplication/cascade.

If I build it with the default values in the Makefile and check build/aie_512x512x512_32x32x32_4c.mlir.prj/input_with_addresses.mlir, I don't see the cascade_flow or configure_cascade op in the MLIR. I attached a .txt file below of the generated MLIR I'm referring to: input_with_addresses.txt.

The kernel code I'm referring to is the one here. The matmul_scalar_cascade_put_only and matmul_scalar_cascade_put_get functions don't use the third argument c, so I'm not sure if it's contributing to the functionality of the kernels.

I also want to mention that the design passes functional verification when I run the host application. I'm trying to figure out how a cascade design is supposed to be written, and I'm just guessing how it's working in the example since I don't see explicit configuration of the cascade stream.

For example, I think in the Versal a cascade stream is set by calling connect<cascade>() with the inputs being the output of a kernel and input of another kernel. I was thinking something similar would be written in code for the Ryzen, which I'm guessing is the cascade_flow op or the configure_cascade op it lowers to.

[jackl-xilinx]

@kurtis-b Yes, you're right, there is no explicit cascade flows in this design and this is on us to create a design to highlight how to do cascade flows in python. The reason the cascade matmul design doesn't show cascade flows and doesn't lower to cascade flows is because it's leveraging the fact the hardware default for cascade flow is down a column. The cascade matmul does exactly that, and as long as the tile putting data onto the cascade stream is above the one getting the data from the cascade stream, it will just work out of the box. This is the default setting for the cascade control mux. This is not helpful for designers looking to be explicit about this connection or to create more customized cascade flows so that's something we need to illustrate in a design example.

For your second question about argument c, you're right in that matmul_scalar_cascade_put_only and matmul_scalar_cascade_put_get do not write to these buffers. However, this does not prevent those buffers from existing and access to these buffers being possibly managed by objfifos locks (acquire, release). Objfifos are still helpful to synchronize compute tiles since locks can be acquired/released when a core is done with the data on the objfifo. In this design, however, those unused outputs are mapped to C_l1l2_buffers that aren't connected to anything but provides an end point to the output port. This is done so we can have the same function port signature for all 3 cores. Only the last row connects that port to an objfifos to move the data onward to mem tiles/ shim tiles as seen here: https://github.com/Xilinx/mlir-aie/blob/main/programming_examples/basic/matrix_multiplication/cascade/cascade_placed.py#L326-L328.

[kurtis-b]

I see, that makes sense. Thanks for clarifying!