Immediate EOI in S-EL2 causes level-triggered IRQ to ping-pong between S-EL2 and S-EL1

[firewof]

Hi,
Our SP developers have reported an IRQ‑related issue. The problem is as follows:
Hafnium receives a pIRQ and signals the S‑EL1 UP SP via a vIRQ; the IRQ is level-triggered and the SP’s handler deasserts the interrupt line. However, Hafnium performs EOI in S-EL2 before switching back to the SP, Link

if (v_intid != SPURIOUS_INTID_OTHER_WORLD) {
	/*
	 * End the interrupt to drop the running priority. It also
	 * deactivates the physical interrupt. If not, the interrupt
	 * could trigger again after resuming current vCPU.
	 */
	plat_interrupts_end_of_interrupt(intid);
}

so as soon as we resume the SP the same pIRQ is immediately re-presented and the CPU returns to Hafnium’s irq_lower handler.

With instrumentation we observe the SP’s PC stuck at the IRQ handler entrypoint (0x93603280) and the system bouncing between Hafnium and the SP without making forward progress.
[3.611642][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.617131][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93600094
[3.622696][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.628344][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93603280
[3.633909][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.639558][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93603280
[3.645122][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.650771][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93603280
[3.656336][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.661985][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93603280
[3.667550][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.673198][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93603280
[3.678764][HF] (0) ERROR: ffa_interrupts_handle_secure_interrupt
[3.684412][HF] (0) ERROR: intid: 681 vm: 0x8001, pc: 0x93603280
Additionally, we found that removing Hafnium’s EOI allows the SP to complete its IRQ handler.

We also noticed a comment stating that delaying EOI would cause a re-trigger after resume current vCPU; for level-triggered IRQs our observation appears to be the opposite—early EOI before the device deasserts the line causes an immediate re-trigger and ping-pong. Do you have any suggestions?

[odeprez]

Hi I believe hafnium has never been tested against level-sensitive interrupts, so this issue is not completely a surprise, and this scenario might require adjustments in the implementation. For starters is there any chance to switch to egde-triggered interrupt as a stop gap? Meanwhile I believe we'll need a round of discussion with architecture team as to whether handling level-sensitive interrupts might require any tweak to FF-A interrupt handling flows.

[firewof]

Thanks for the quick response. We’ve temporarily switched the source to edge‑triggered as a stopgap, so this isn’t urgent right now.

We also tried deferring EOI until the SP issues HF_INTERRUPT_DEACTIVATE. This works unless the pIRQ is taken on one PE and the vIRQ is delivered to the SP running on a different PE; in that cross‑PE case the EOI on the other core doesn’t clear the Active state, so deactivation fails. Sharing this observation for your design considerations.