From my experiments in #939 it seems that with the ESP32 I-cache miss is a microsecond-level (several thousand cycles) penalty. In ESP-IDF they have a Kconfig
based mechanism that allows putting devices drivers in the IRAM. We should have the same ability in order to enable timing-sensitive applications.