|
| 1 | +======================================== |
| 2 | +Signaling Events from Interrupt Handlers |
| 3 | +======================================== |
| 4 | + |
| 5 | +.. warning:: Migrated from |
| 6 | + https://cwiki.apache.org/confluence/display/NUTTX/Signaling+Events+from+Interrupt+Handlers |
| 7 | + |
| 8 | +Best way to wake multiple threads from interrupt? |
| 9 | +================================================= |
| 10 | + |
| 11 | + I want to make a character device driver that passes the same data to |
| 12 | + all tasks that are reading it. It is not so important whether the data |
| 13 | + is queued or if just latest sample is retrieved. Problem is just how to |
| 14 | + wake up the waiting threads. |
| 15 | + |
| 16 | +At the most primitive level, a thread can be waiting for a semaphore, a signal, |
| 17 | +or a message queue (not empty or not full). Then there are higher |
| 18 | +level wrappers around these like mutexes, semaphores, poll waits, |
| 19 | +etc. But under the hood those are the three fundamental wait |
| 20 | +mechanisms. Any could be used to accomplish what you want. |
| 21 | + |
| 22 | +In NuttX, some additional effort was put into the design of the signalling |
| 23 | +side of each of the IPCs so that they could be easily used by interrupts |
| 24 | +handlers. This behavior is unique to NuttX; POSIX says nothing about |
| 25 | +interrupt handlers. As a result, we will be talking about primarily |
| 26 | +non-portable OS interfaces. |
| 27 | + |
| 28 | + So far I've considered the following options: |
| 29 | + |
| 30 | +And you basically have gone through the list of wait mechanisms: |
| 31 | + |
| 32 | +Message Queues |
| 33 | +============== |
| 34 | + |
| 35 | + 1) Open a message queue when the device is opened (a new queue for each |
| 36 | + task) and keep them in a list. Post to a non-blocking endpoint of these |
| 37 | + queues in the ISR. Read from a blocking endpoint in the device ``read()``. |
| 38 | + I would need to generate names for the message queues, as there doesn't |
| 39 | + seem to be anonymous message queues? |
| 40 | + |
| 41 | +When you start a project. It is a good idea to decide upon a common IPC |
| 42 | +mechanism to base your design on. POSIX message queues are one good |
| 43 | +choice to do that: Assign each thread a message queue and the ``main()`` |
| 44 | +of each thread simply waits on the message queue. It is a good |
| 45 | +architecture and used frequently. |
| 46 | + |
| 47 | +However, I would probably avoid creating a lot of message queues just |
| 48 | +to support the interrupt level signaling. There are other ways to do |
| 49 | +that that do not use so much memory. So, if you have message queues, |
| 50 | +use them. If not, keep it simple. |
| 51 | + |
| 52 | +In this case, your waiting task will block on a call to ``mq_receive()`` |
| 53 | +until a message is received. It will then wake up and can process |
| 54 | +the message. In the interrupt handler, it will call ``mq_send()`` when |
| 55 | +an event of interest occurs which will, in turn, wake up the waiting |
| 56 | +task. |
| 57 | + |
| 58 | +Advantages of the use of message queues in this case are that 1) you |
| 59 | +can pass quite a lot of data in the message, and 2) it integrates |
| 60 | +well in a message-based application architecture. A disadvantage |
| 61 | +is that there is a limitation on the number of messages that can be |
| 62 | +sent from an interrupt handler so it is possible to get data overrun |
| 63 | +conditions, that is, more interrupt events may be received than can |
| 64 | +be reported with the available messages. |
| 65 | + |
| 66 | +This limitation is due to the fact that you cannot allocate memory |
| 67 | +dynamically from an interrupt handler. Instead, interrupt handlers |
| 68 | +are limited to the use of pre-allocated messages. The number of |
| 69 | +pre-allocated messages is given by ``CONFIG_PREALLOC_MQ_MSGS`` + 8. |
| 70 | +The ``CONFIG_PREALLOC_MQ_MSGS`` can be used either by normal tasking |
| 71 | +logic or by interrupt level logic. The extra eight are an emergency |
| 72 | +pool for interrupt handling logic only (that value is not currently |
| 73 | +configurable). |
| 74 | + |
| 75 | +If the task logic consumes all of the ``CONFIG_PREALLOC_MQ_MSGS`` messages, it |
| 76 | +will fall back to dynamically allocating messages at some cost to |
| 77 | +performance and deterministic behavior. |
| 78 | + |
| 79 | +If the interrupt level consumes all of the ``CONFIG_PREALLOC_MQ_MSGS`` |
| 80 | +messages, it will fall back and use the emergency pool of 8 |
| 81 | +pre-allocated messages. If those are also exhausted, then the message |
| 82 | +will not be sent and an interrupt is effectively lost. |
| 83 | + |
| 84 | +Semaphores |
| 85 | +========== |
| 86 | + |
| 87 | + 2) Allocate a semaphore per each device open and keep them in a list. |
| 88 | + Post the semaphores when new data is available in a shared buffer. |
| 89 | + Read the data inside ``sched_lock()``. |
| 90 | + |
| 91 | +If you don't have an architecture that uses message queues, and all of |
| 92 | +these threads are waiting only for the interrupt event and nothing else, |
| 93 | +then signaling semaphores would work fine too. You are basically using |
| 94 | +semaphores as condition variables in this case so you do have to be careful. |
| 95 | + |
| 96 | +NOTE: You do not need multiple semaphores. You can do this with a single |
| 97 | +semaphore. If the semaphore is used for this purpose then you initialize |
| 98 | +it to zero: |
| 99 | + |
| 100 | +.. code-block:: c |
| 101 | +
|
| 102 | + sem_init(&sem, 0, 0); |
| 103 | + sem_setprotocol(&sem, SEM_PRIO_NONE); |
| 104 | +
|
| 105 | +``sem_setprotocol()`` is a non-standard NuttX function that should be called |
| 106 | +immediately after the ``sem_init()``. The effect of this function call is to |
| 107 | +disable priority inheritance for that specific semaphore. There should |
| 108 | +then be no priority inheritance operations on this semaphore that is |
| 109 | +used for signaling. See `Signaling Semaphores and Priority Inheritance |
| 110 | +<https://cwiki.apache.org/confluence/display/NUTTX/Signaling+Semaphores+and+Priority+Inheritance>`_ |
| 111 | +for further information. |
| 112 | + |
| 113 | +Since the semaphore is initialized to zero, each time that a thread joins |
| 114 | +the group of waiting threads, the count is decremented. So a simple loop |
| 115 | +like this would wake up all waiting threads: |
| 116 | + |
| 117 | +.. code-block:: c |
| 118 | +
|
| 119 | + int svalue; |
| 120 | + int ret; |
| 121 | + |
| 122 | + for (; ; ) |
| 123 | + { |
| 124 | + ret = sem_getvalue(&sem, &svalue); |
| 125 | + if (svalue < 0) |
| 126 | + { |
| 127 | + sem_post(&sem); |
| 128 | + } |
| 129 | + else |
| 130 | + { |
| 131 | + break; |
| 132 | + } |
| 133 | + } |
| 134 | +
|
| 135 | +NOTE: This use of ``sem_getvalue()`` is not portable. In many environments, |
| 136 | +``sem_getvalue()`` will not return negative values if there are waiters on |
| 137 | +the semaphore. |
| 138 | + |
| 139 | +The above code snippet is essentially what the NuttX |
| 140 | +``pthread_cond_broadcast()`` does (see `nuttx/sched/pthread_condbroadcast.c <https://github.com/apache/nuttx/blob/master/sched/pthread/pthread_condbroadcast.c>`_). |
| 141 | +In NuttX condition variables are really just wrappers around semaphores |
| 142 | +that give them a few new properties. You could even call |
| 143 | +``pthread_cond_broadcast()`` from an interrupt handler: See |
| 144 | +http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_cond_signal.html |
| 145 | +for usage information. |
| 146 | + |
| 147 | +Neither of the above mechanisms are portable uses of these interfaces. |
| 148 | +However, there is no portable interface for communicating directly with |
| 149 | +interrupt handlers. |
| 150 | + |
| 151 | +If you want to signal a single waiting thread, there are simpler things |
| 152 | +you an do. In the waiting task: |
| 153 | + |
| 154 | +.. code-block:: c |
| 155 | +
|
| 156 | + semt_t g_mysemaphore; |
| 157 | + volatile bool g_waiting; |
| 158 | + ... |
| 159 | + |
| 160 | + sem_init(&g_mysemaphore); |
| 161 | + sem_setprotocol(&g_mysemaphore, SEM_PRIO_NONE); |
| 162 | + ... |
| 163 | + |
| 164 | + flags = enter_critical_section(); |
| 165 | + g_waiting = true; |
| 166 | + while (g_waiting) |
| 167 | + { |
| 168 | + ret = sem_wait(&g_mysemaphore); |
| 169 | + ... handler errors ... |
| 170 | + } |
| 171 | + |
| 172 | + leave_critical_section(flags); |
| 173 | +
|
| 174 | +In the above code snippet, interrupts are disabled to set and test |
| 175 | +``g_waiting``. Interrupts will, of course, be re-enabled automatically |
| 176 | +and atomically while the task is waiting for the interrupt event. |
| 177 | + |
| 178 | +Then in the interrupt handler |
| 179 | + |
| 180 | +.. code-block:: c |
| 181 | +
|
| 182 | + extern semt_t g_mysemaphore; |
| 183 | + extern volatile bool g_waiting; |
| 184 | + ... |
| 185 | + |
| 186 | + if (g_waiting) |
| 187 | + { |
| 188 | + g_waiting = false; |
| 189 | + sem_post(&g_mysemaphore); |
| 190 | + } |
| 191 | +
|
| 192 | +An integer type counter could also be used instead of a type bool to |
| 193 | +support multiple waitings. In that case, this is equivalent to the |
| 194 | +case above using ``sem_getvalue()`` but does not depend on non-portable |
| 195 | +properties of ``sem_getvalue()``. |
| 196 | + |
| 197 | +NOTE: There is possibility of improper interactions between the |
| 198 | +semaphore when it is used for signaling and priority inheritance. |
| 199 | +In this case, you should disable priority inheritance on the |
| 200 | +signaling semaphore using ``sem_setprotocol(SEM_PRIO_NONE)``. See `Signaling Semaphores and Priority Inheritance |
| 201 | +<https://cwiki.apache.org/confluence/display/NUTTX/Signaling+Semaphores+and+Priority+Inheritance>`_ |
| 202 | +for further information. |
| 203 | + |
| 204 | +Signals |
| 205 | +======= |
| 206 | + |
| 207 | + 3) Store the thread id's in a list when ``read()`` is called. Wake up the |
| 208 | + threads using ``sigqueue()``. Read the data from a shared buffer |
| 209 | + inside ``sched_lock()``. |
| 210 | + |
| 211 | +Signals would work fine too. Signals have a side-effect that is sometimes |
| 212 | +helpful and sometimes a pain in the butt: They cause almost all kinds of |
| 213 | +waits (``read()``, ``sem_wait()``, etc.) to wake up and return an error with |
| 214 | +``errno=EINTR``. |
| 215 | + |
| 216 | +That is sometimes helpful because you can wake up a ``recv()`` or a ``read()`` |
| 217 | +etc., detect the event that generated the signal, and do something |
| 218 | +about it. It is sometimes a pain because you have to remember to |
| 219 | +handle the ``EINTR`` return value even when you don't care about it. |
| 220 | + |
| 221 | +The POSIX signal definition includes some support that would make this |
| 222 | +easier for you. This support is not currently implemented in NuttX. |
| 223 | +The ``kill()`` interface for example |
| 224 | +(http://pubs.opengroup.org/onlinepubs/009695399/functions/kill.html) |
| 225 | +supports this behavior: |
| 226 | + |
| 227 | +"If pid is 0, sig will be sent to all processes (excluding an unspecified |
| 228 | +set of system processes) whose process group ID is equal to the process |
| 229 | +group ID of the sender, and for which the process has permission to send |
| 230 | +a signal. |
| 231 | + |
| 232 | +"If pid is -1, sig will be sent to all processes (excluding an unspecified |
| 233 | +set of system processes) for which the process has permission to send that |
| 234 | +signal." |
| 235 | + |
| 236 | +"If pid is negative, but not -1, sig will be sent to all processes (excluding |
| 237 | +an unspecified set of system processes) whose process group ID is equal to |
| 238 | +the absolute value of pid, and for which the process has permission to send |
| 239 | +a signal." |
| 240 | + |
| 241 | +NuttX does not currently support process groups. But that might be a good |
| 242 | +RTOS extension. If you and others think that would be useful I could |
| 243 | +probably add the basics of such a feature in a day or so. |
| 244 | + |
| 245 | +poll() |
| 246 | +====== |
| 247 | + |
| 248 | + Is there some better way that I haven't discovered? |
| 249 | + |
| 250 | +The obvious thing that you did not mention is ``poll()``. See |
| 251 | +http://pubs.opengroup.org/onlinepubs/009695399/functions/poll.html . |
| 252 | +Since you are writing a device driver, support for the ``poll()`` method |
| 253 | +in your driver seems to be the natural solution. See the ``drivers/`` |
| 254 | +directory for many examples, ``drivers/pipes/pipe_common.c`` for one. |
| 255 | +Each thread could simply wait on ``poll()``; when the event occurs the |
| 256 | +driver could then wake up the set of waiters. Under the hood, this |
| 257 | +is again just a set of ``sem_post``'s. But it is also a very standard |
| 258 | +mechanism. |
| 259 | + |
| 260 | +In your case, the semantics of ``poll()`` might have to be bent just a |
| 261 | +little. You might have to bend the meaning of some of the event |
| 262 | +flags since they are all focused on data I/O events. |
| 263 | + |
| 264 | +Another creative use of ``poll()`` for use in cases like this: |
| 265 | + |
| 266 | + That would be something great! PX4 project has that implemented somehow |
| 267 | + (in C++), so maybe - if license permits - it could be ported to NuttX in |
| 268 | + no time? |
| 269 | + |
| 270 | + https://pixhawk.ethz.ch/px4/dev/shared_object_communication |
| 271 | + |
| 272 | +I don't know a lot about this, but it might be worth looking into |
| 273 | +if it matches your need. |
0 commit comments