Add documentation about PDA, C-RORC

Author: PascalBoeschoten

README.md

@@ -100,10 +100,10 @@ DMA can be paused and resumed at any time using `stopDma()` and `startDma()`
 
 ## Note for when bad things happen
 The driver uses some files in shared memory:
-* `/dev/shm/alice_o2/rorc/[PCI address]/channel_[channel number]/fifo` - For card FIFOs
-* `/dev/shm/alice_o2/rorc/[PCI address]/channel_[channel number]/.lock` - For locking channels
-* `/dev/shm/sem.alice_o2_rorc_[PCI address]_channel_[channel number].mutex` - For locking channels
-* `/var/lib/hugetlbfs/global/pagesize-[page size]/rorc-dma-bench_id=[PCI address]_chan_[channel number]` - For card benchmark DMA buffers
+* `/dev/shm/AliceO2_RoC_[PCI address]_Channel_[channel number]_fifo` - For card FIFOs
+* `/dev/shm/AliceO2_RoC_[PCI address]_Channel_[channel number].lock` - For locking channels
+* `/dev/shm/sem.AliceO2_RoC_[PCI address]_Channel_[channel number]_Mutex` - For locking channels
+* `/var/lib/hugetlbfs/global/pagesize-[page size]/roc-dma-bench_id=[PCI address]_chan_[channel number]` - For roc-bench-dma buffers
 
 If the process crashes badly (such as with a segfault), it may be necessary to clean up the mutex manually, either by
 deleting with `rm` or by using the `roc-channel-cleanup` utility. 
@@ -186,9 +186,12 @@ Resets a card channel
 
 ### roc-run-script
 *Deprecated, see section "Python interface"*
-
 Run a Python script that can use a simple interface to use the library.
 
+### roc-setup-hugetlbfs
+Setup hugetlbfs directories & mounts. If using hugepages, should be run once per boot.
+
+
 
 Exceptions
 -------------------
@@ -256,7 +259,7 @@ If PDA is not detected on the system, only a dummy implementation of the interfa
 
 4. Optionally, insert kernel module. If the utilities are run as root, PDA will do this automatically.
   ~~~
-  modprobe uio\_pci\_dma
+  modprobe uio_pci_dma
   ~~~
 
 ### Hugepages
@@ -340,10 +343,8 @@ The issue has occurred on Dell R720 servers.
 
 Permissions
 -------------------
-The library must be run either by root users, or users part of the group 'pda'. In case of 'pda' group users, make sure
-the `/dev/shm/alice_o2` and `/mnt/hugetlbfs/alice_o2` directories have sufficient permissions that allow those 
-users to create/read/write files.
-Also, the PDA kernel module must be inserted as root in any case.
+The library must be run either by root users, or users part of the group 'pda'.
+The PDA kernel module must be inserted as root in any case.
 
 
 ALICE Low-level Front-end (ALF) DIM Server

src/Crorc/CrorcDmaChannel.cxx

@@ -81,6 +81,8 @@ CrorcDmaChannel::~CrorcDmaChannel()
 
 void CrorcDmaChannel::deviceStartDma()
 {
+  // With the C-RORC, we can't start DMA until we have enough memory to cover 128 DMA pages (which should be covered by
+  // 1 superpage). So we set the "pending DMA start" state and actually start once a superpage has been pushed.
   log("DMA start deferred until superpage available");
 
   mFifoBack = 0;
@@ -285,6 +287,7 @@ void CrorcDmaChannel::pushSuperpage(Superpage superpage)
   if (!Utilities::isMultiple(superpage.getSize(), MIN_SIZE)) {
     BOOST_THROW_EXCEPTION(CrorcException()
         << ErrorInfo::Message("Could not enqueue superpage, C-RORC backend requires superpage size multiple of 1 MiB"));
+    // We require 1 MiB because this fits 128 8KiB DMA pages (see deviceStartDma() for why we need that)
   }
 
   SuperpageQueueEntry entry;

src/Pda/README.md

@@ -30,4 +30,8 @@ The obstacle to this is that some of the exceptions thrown are somewhat ReadoutC
 For example, in `PdaDmaBuffer.cxx`, one of the exceptions presents the message:
 *"Scatter-gather node smaller than 2 MiB (minimum hugepage size. This means the IOMMU is off and the buffer is not 
 backed by hugepages - an unsupported buffer configuration."*
-While this is helpful in the context of the way it is used in ReadoutCard, it may not apply in different situations.
+While this is helpful in the context of the way it is used in ReadoutCard, it may not apply in different situations.
+
+## Locking
+If multiple DMA buffers are concurrently created/destroyed, the PDA kernel module will lock up, requiring a reboot.  
+To prevent this from happening, the global lock `PdaLock` is used in `PdaDmaBuffer.cxx` and `Driver.cxx`.