|
| 1 | +P2P FPGA to FPGA |
| 2 | +================ |
| 3 | + |
| 4 | +PCIe peer-to-peer communication (P2P) is a PCIe feature which enables |
| 5 | +two PCIe devices to directly transfer data between each other without |
| 6 | +using host RAM as a temporary storage. The latest version of Vitis PCIe |
| 7 | +platforms support P2P feature via PCIe Resizable BAR Capability |
| 8 | + |
| 9 | +In this Example we highlight the data transfer between FPGA and FPGA |
| 10 | +devices. The current design support P2P between 2 similar FPGA devices. |
| 11 | +User can update the makefile if need to run P2P between different FPGA |
| 12 | +cards. |
| 13 | + |
| 14 | +Enable/Disable P2P |
| 15 | +~~~~~~~~~~~~~~~~~~ |
| 16 | + |
| 17 | +XRT xbutil is used to enable/disable P2P feature and check current |
| 18 | +configuration. P2P configuration is persistent across warm reboot. |
| 19 | +Enabling or disabling P2P requires root privilege. |
| 20 | + |
| 21 | +Enabling P2P after cold boot is likely to fail because it resizes an |
| 22 | +existing P2P PCIe BAR to a large size and usually Linux will not reserve |
| 23 | +large IO memory for the PCIe bridges. A warm reboot is required in this |
| 24 | +scenario after which BIOS and Linux will reassign the required expanded |
| 25 | +IO memory resource for P2P BAR. Disabling P2P takes effect immediately. |
| 26 | +Currently XRT does not check if the P2P memory is in use. Administrator |
| 27 | +needs to make sure P2P is not in use before disabling it. The result of |
| 28 | +disabling P2P while it is in use is undefined. |
| 29 | + |
| 30 | +There are three possible values for P2P Enabled field : |
| 31 | + |
| 32 | ++-------------------------+--------------------------------------------+ |
| 33 | +| Value | Remarks | |
| 34 | ++=========================+============================================+ |
| 35 | +| True | P2P is Enabled | |
| 36 | ++-------------------------+--------------------------------------------+ |
| 37 | +| False | P2P is Disabled | |
| 38 | ++-------------------------+--------------------------------------------+ |
| 39 | +| no iomem | P2P is enabled in device but system could | |
| 40 | +| | not allocate IO memory, warm reboot is | |
| 41 | +| | needed | |
| 42 | ++-------------------------+--------------------------------------------+ |
| 43 | + |
| 44 | +Enable P2P |
| 45 | +^^^^^^^^^^ |
| 46 | + |
| 47 | +Enable P2P after power up sequence. |
| 48 | + |
| 49 | +:: |
| 50 | + |
| 51 | + # xbutil p2p --enable |
| 52 | + ERROR: resoure busy, please try warm reboot |
| 53 | + # xbutil query |
| 54 | + ... |
| 55 | + PCIe DMA chan(bidir) MIG Calibrated P2P Enabled |
| 56 | + GEN 3x16 2 true no iomem |
| 57 | + # reboot |
| 58 | + ... |
| 59 | + # xbutil query |
| 60 | + ... |
| 61 | + PCIe DMA chan(bidir) MIG Calibrated P2P Enabled |
| 62 | + GEN 3x16 2 true true |
| 63 | + ... |
| 64 | + |
| 65 | +Disable P2P |
| 66 | +^^^^^^^^^^^ |
| 67 | + |
| 68 | +:: |
| 69 | + |
| 70 | + # xbutil p2p --disable |
| 71 | + # xbutil query |
| 72 | + ... |
| 73 | + PCIe DMA chan(bidir) MIG Calibrated P2P Enabled |
| 74 | + GEN 3x16 2 true false |
| 75 | + ... |
0 commit comments