Inaccurate deepsleep with CONFIG_ESP32_RTC_CLOCK_SOURCE_EXTERNAL_CRYSTAL

[ekspla]

MPY: 1.21.0
ESP-IDF: 5.0.2
ESP32-Devkit-C (ESP32-WROOM-32E)

I am planning to use esp32 as a battery powered logger, where ntpdate via wifi is not available.

According to the following comments,
ESP32 - External 32.768 Xtal #3764

The header pins at GPIO 32 and 33 were removed, and a 32.768 kHz crystal with a pair of 11 pF chip capacitors were soldered.
I modified the files and built/installed the firmware to use the external crystal.

ESP32_GENERIC_RTC_EXT_CRYS/sdkconfig

CONFIG_ESP32_RTC_CLOCK_SOURCE_EXTERNAL_CRYSTAL=y

mpconfigboard.cmake

set(SDKCONFIG_DEFAULTS
    boards/sdkconfig.base
    boards/sdkconfig.ble
    boards/ESP32_GENERIC_RTC_EXT_CRYS/sdkconfig
)
 ...

The problems I have are inaccurate duration of deepsleep and the timestamp after the sleep, although
machine.RTC.datetime() and time.localtime() without sleep showed only a few second of error in one week.

The durations of 10 minutes deepsleep differs by several seconds; time stamps after the sleeps were advanced
by a few seconds; This might be okay though ...

The duration of 1 hour deepsleep was 14 seconds less than what was expected.

I checked this by using additional I2C RTC (PCF8563) for the reference (please see below), whose clock was accurate within 5 ppm.

I have following questions,

1. Is there an easy way to check internal RTC clock frequency?
   I'm almost sure that the probe to pick up the clock affects the RTC oscillator.

2. Am I doing something wrong in building the firmware?

3. Is it impossible to expect an accurate internal RTC after the deepsleep?

boot.py

import machine, time, pcf8563

rtc = machine.RTC()
i2c = machine.I2C(0)
pcf_rtc = pcf8563.PCF8563(i2c)

print(f'machine.RTC {rtc.datetime()}')
print(f'time.localtime {time.localtime()}')
print(f'PCF8563.datetime {pcf_rtc.datetime()}')

Test results

>>> pcf_rtc.write_now()                 # Reset timestamp
>>> pcf_rtc.datetime(); rtc.datetime()
(23, 11, 21, 1, 9, 15, 36)              # This is reference RTC.
(2023, 11, 21, 1, 9, 15, 36, 391356)    # This is internal RTC, almost the same timestamp.
>>> machine.deepsleep(600_000)          # 10 min. deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 21, 1, 9, 25, 46, 983607)    # 09:25:46.983607, ~+2 sec advanced from ext. RTC.
time.localtime (2023, 11, 21, 9, 25, 46, 1, 325)
PCF8563.datetime (23, 11, 21, 1, 9, 25, 44)         # 10 min + ~8 sec.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> pcf_rtc.write_now()                 # Reset timestamp
>>> pcf_rtc.datetime(); rtc.datetime()
(23, 11, 21, 1, 9, 39, 52)              # This is reference RTC.
(2023, 11, 21, 1, 9, 39, 52, 847023)    # Almost the same
>>> machine.deepsleep(600_000)          # 10 min. deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 21, 1, 9, 49, 54, 899149)    # 09:49:54.899149, ~+2 sec advanced.
time.localtime (2023, 11, 21, 9, 49, 54, 1, 325)
PCF8563.datetime (23, 11, 21, 1, 9, 49, 52)         # Exactly 10 min.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>> pcf_rtc.write_now()                  # Reset timestamp
>>> pcf_rtc.datetime(); rtc.datetime()
(23, 11, 21, 1, 9, 53, 6)                # This is reference RTC.
(2023, 11, 21, 1, 9, 53, 7, 172769)      # 09:53:07.172769, ~+1 sec from ext. RTC.
>>> machine.deepsleep(600_000)           # 10 min. deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 21, 1, 10, 3, 9, 334939)    # 10:03:09.334939, ~+2 sec advanced.
time.localtime (2023, 11, 21, 10, 3, 9, 1, 325)
PCF8563.datetime (23, 11, 21, 1, 10, 3, 6)         # Exactly 10 min.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> pcf_rtc.write_now()                  # Reset timestamp
>>> pcf_rtc.datetime(); rtc.datetime()
(23, 11, 21, 1, 10, 12, 13)              # This is reference RTC.
(2023, 11, 21, 1, 10, 12, 13, 358356)    # 10:12:13.358356, almost the same
>>> machine.deepsleep(3600_000)          # 1 hr. deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 21, 1, 11, 12, 15, 881223)    # 11:12:15.881223
time.localtime (2023, 11, 21, 11, 12, 15, 1, 325)
PCF8563.datetime (23, 11, 21, 1, 11, 11, 59)         # 11:11:59, ~14 sec less than 1 hr.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>

[karfas]

I would assume the ESP32 doesn't use your XTAL (but have no idea how to check this).

A quick google search gave me

• https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/system_time.html
• https://www.espressif.com/sites/default/files/documentation/esp32_hardware_design_guidelines_en.pdf#page=15
• https://climbers.net/sbc/esp32-accurate-clock-sleep-ntp/ (calibrate drift using NTP)

The guy who wrote the third link got good results with periodic calibration of the clock.

[ekspla]

@karfas Thank you for the comments. In response to his comment:

I read the second document he cited once again.

After reducing the bias resistor by half (10 M // 10 M = 5 M Ohm), the RTC oscillator in my ESP32 seems working properly (see below).
Though tweaking parameters without knowing the details inside of the oscillator was a bad idea, this was the way that I could do...

One of my questions (1) still remains open. Maybe, I need a good active probe of high Z/low C, but I have a bad experience in using it tens of years ago when I was a student.

>>> pcf_rtc.write_now()                  # Reset timestamp
>>> pcf_rtc.datetime();rtc.datetime()
(23, 11, 22, 2, 7, 44, 48)               # This is reference RTC.
(2023, 11, 22, 2, 7, 44, 49, 484739)     # Almost the same, ~ +1 sec.
>>> machine.deepsleep(600_000)           # 10 min deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 22, 2, 7, 54, 52, 395130)  # Seems o.k. Almost the same, ~ +1 sec.
time.localtime (2023, 11, 22, 7, 54, 52, 2, 326)
PCF8563.datetime (23, 11, 22, 2, 7, 54, 51)       # 10 min + 3 sec.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> pcf_rtc.datetime();rtc.datetime()
(23, 11, 22, 2, 7, 59, 34)               # This is reference RTC.
(2023, 11, 22, 2, 7, 59, 34, 888399)     # Almost the same timestamp.
>>> machine.deepsleep(600_000)           # 10 min deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 22, 2, 8, 9, 36, 347989)  # Seems o.k. Almost the same, ~ +1 sec.
time.localtime (2023, 11, 22, 8, 9, 36, 2, 326)
PCF8563.datetime (23, 11, 22, 2, 8, 9, 35)       # 10 min + 1 sec.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> pcf_rtc.datetime();rtc.datetime()
(23, 11, 22, 2, 8, 15, 26)               # This is reference RTC.
(2023, 11, 22, 2, 8, 15, 27, 251164)     # Almost the same, ~ +1 sec.
>>> machine.deepsleep(600_000)           # 10 min deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 22, 2, 8, 25, 28, 840764)  # Seems o.k. Almost the same.
time.localtime (2023, 11, 22, 8, 25, 28, 2, 326)
PCF8563.datetime (23, 11, 22, 2, 8, 25, 28)       # 10 min + 2 sec.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> pcf_rtc.datetime();rtc.datetime()
(23, 11, 22, 2, 8, 27, 32)               # This is reference RTC.
(2023, 11, 22, 2, 8, 27, 33, 273935)     # Almost the same, ~ +1 sec.
>>> machine.deepsleep(3600_000)          # 1 hr deepsleep
ets Jul 29 2019 12:21:46

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4728
load:0x40078000,len:15400
load:0x40080400,len:3376
entry 0x400805cc
machine.RTC (2023, 11, 22, 2, 9, 27, 34, 713491)  # Seems o.k. Almost the same.
time.localtime (2023, 11, 22, 9, 27, 34, 2, 326)
PCF8563.datetime (23, 11, 22, 2, 9, 27, 33)       # 1 hr. + 1 sec.
MicroPython v1.21.0 on 2023-11-13; Generic ESP32 module with ESP32
Type "help()" for more information.
>>>

[kjm1102]

Strikes me as great shame that there isn't a snooze mode that keeps the esp32 crystal oscillator running. This is the source for the RTC when the chip is awake & explains why RTC timekeeping goes to the dogs when we deepsleep the chip. I've never had much success poking around crystal oscillators with a x10 CRO probe without frequency pulling. Sometimes, if you're lucky, you can acquire a signal with an inductive pickup but it's often from the downstream divider rather than the raw xtal frequency.