[Bug]: CPU usage reaches over 100% before freezing host computer

[andrew-chang-dewitt]

Controller Version

v5.13

Describe the Bug

After starting docker container, the below process begins to eat up CPU until it eventually passes 100% and locking up the entire computer. The computer ends up requiring power be cut manually to restart it.

andrew@jane: ~
$ ps -aux
...
508         1476 57.1  5.9 3461088 51520 ?       Ssl  15:20   0:03 /usr/bin/java -server -Xms128m -Xmx1024m -XX:MaxHeapFreeRatio=60 -
...

During this time, the controller is never available on <host>:8088 or <host>:8043 and the docker container never appears to move past "starting" according to the healthcheck:

andrew@jane: ~
$ docker ps
CONTAINER ID   IMAGE                                          COMMAND                  CREATED          STATUS                           PORTS     NAMES
74f1921df171   mbentley/omada-controller                      "/entrypoint.sh /usr…"   15 minutes ago   Up 1 second (health: starting)             omada-controller

I've been trying things like limiting the docker container's memory or cpu w/ --cpus & -m (see run command below) as well as modifying the command run at the end of the Dockerfile as suggested in #110, but can't seem to get anything to work.

Expected Behavior

Docker container should start omada controller successfully, eventually making the web app accessible at <host>:8088. Ideally, it should do this without such high CPU usage.

Any help getting this to run on my Raspberry Pi 3B+ would be very much appreciated as I'd much prefer this to buying the hardware controller or running the software controller on my full-size desktop that I use for a lot of other things and restart regularly.

Steps to Reproduce

1. Install Ubuntu 23.10 on a Raspberry Pi 3B+
2. Install docker engine
3. Start container w/ below command

How You're Launching the Container

docker run -d \
  --name omada-controller \
  --network=host \
  --cpus 1 \
  -m 512mb \
  -e MANAGE_HTTP_PORT=8088 \
  -e MANAGE_HTTPS_PORT=8043 \
  -e PORTAL_HTTP_PORT=8088 \
  -e PORTAL_HTTPS_PORT=8843 \
  -e SHOW_SERVER_LOGS=true \
  -e SHOW_MONGODB_LOGS=false \
  -e SSL_CERT_NAME="tls.crt" \
  -e SSL_KEY_NAME="tls.key" \
  -e TZ=Etc/UTC \
  -v /data/omada/data:/opt/tplink/EAPController/data \
  -v /data/omada/logs:/opt/tplink/EAPController/logs \
  mbentley/omada-controller

Container Logs

andrew@jane: ~
$ docker logs omada-controller >& output.log

andrew@jane: ~
$ cat output.log
INFO: Validating user/group (omada:omada) exists with correct UID/GID (508:508)
INFO: Group (omada) doesn't exist; creating
INFO: User (omada) doesn't exist; creating
INFO: Time zone set to 'Etc/UTC'
INFO: Value of 'manage.http.port' already set to 8088 in omada.properties
INFO: Value of 'manage.https.port' already set to 8043 in omada.properties
INFO: Value of 'portal.http.port' already set to 8088 in omada.properties
INFO: Value of 'portal.https.port' already set to 8843 in omada.properties
INFO: Value of 'port.adopt.v1' already set to 29812 in omada.properties
INFO: Value of 'port.app.discovery' already set to 27001 in omada.properties
INFO: Value of 'port.upgrade.v1' already set to 29813 in omada.properties
INFO: Value of 'port.manager.v1' already set to 29811 in omada.properties
INFO: Value of 'port.manager.v2' already set to 29814 in omada.properties
INFO: Value of 'port.discovery' already set to 29810 in omada.properties
INFO: Value of 'port.transfer.v2' already set to 29815 in omada.properties
INFO: Value of 'port.rtty' already set to 29816 in omada.properties
INFO: Value of 'mongo.external' already set to false in omada.properties
INFO: Value of 'eap.mongod.uri' already set to mongodb://127.0.0.1:27217/omada in omada.properties
INFO: Report HTML directory missing; extracting backup to '/opt/tplink/EAPController/data/html'
html/
html/logo.png
html/echarts.min.js
html/Omada.png
INFO: Report PDF directory missing; creating '/opt/tplink/EAPController/data/pdf'
WARN: Ownership not set correctly on '/opt/tplink/EAPController/properties'; setting correct ownership (omada:omada)
INFO: Database directory missing; creating '/opt/tplink/EAPController/data/db'
done
INFO: Version check passed; image version (5.13.23) >= the last version ran (0.0.0); writing image version to last ran file...
INFO: userland/kernel check passed
INFO: Starting Omada Controller as user omada
tail: cannot open '/opt/tplink/EAPController/logs/server.log' for reading: No such file or directory
tail: '/opt/tplink/EAPController/logs/server.log' has appeared;  following new file
02-01-2024 21:08:40.586 INFO [main] [] c.t.s.o.s.OmadaLinuxMain(): record: start the omada controller
02-01-2024 21:08:40.682 INFO [main] [] c.t.s.o.s.OmadaLinuxMain(): record: set property finished
02-01-2024 21:08:40.772 INFO [main] [] c.t.s.o.s.OmadaLinuxMain(): record: configure log finished
02-01-2024 21:08:40.861 INFO [main] [] c.t.s.o.s.OmadaBootstrap(): record: bootstrap prepare
02-01-2024 21:08:40.993 INFO [log4j-thread] [] c.t.s.o.c.o.a.b(): success to load configuration omada.properties
02-01-2024 21:08:41.059 INFO [log4j-thread] [] c.t.s.o.c.o.OmadacType(): omadacType: Local Controller
02-01-2024 21:08:42.081 INFO [main] [] c.t.s.o.s.OmadaBootstrap(): going to start local mongod.
02-01-2024 21:08:44.890 INFO [main] [] c.t.s.o.s.s.b(): mongodb process id is 169
02-01-2024 21:08:44.901 ERROR [main] [] c.t.s.f.c.FacadeUtils(): facadeMsgEnable is not enable, msg: Mongo DB server started
02-01-2024 21:08:44.905 INFO [main] [] c.t.s.o.s.s.b(): Mongo DB server started
02-01-2024 21:08:48.673 WARN [main] [] c.t.s.o.s.OmadaBootstrap(): Valid keystore is missing. Generating one ...
02-01-2024 21:09:02.558 INFO [main] [] c.t.s.o.s.OmadaBootstrap(): Certificate generated!
02-01-2024 21:09:07.295 INFO [main] [] c.t.s.o.s.OmadaBootstrap(): record: bootstrap record finished
02-01-2024 21:09:07.295 INFO [main] [] c.t.s.o.s.OmadaBootstrap(): record: bootstrap startup
02-01-2024 21:09:08.485 INFO [main] [] c.t.s.o.s.OmadaBootstrap(): record: start run omada tasks
02-01-2024 21:09:08.489 INFO [main] [] c.t.s.o.s.t.SpringBootStartUpTask(): record: task SpringBootStartupTask start

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::               (v2.7.13)

02-01-2024 21:09:21.179 INFO [main] [] c.t.s.o.s.OmadaLinuxMain(): Starting OmadaLinuxMain v5.13.23 using Java 17.0.9 on jane with PID 1 (/opt/tplink/EAPController/lib/local-starter-5.13.23.jar started by omada in /opt/tplink/EAPController/lib)
02-01-2024 21:09:21.202 INFO [main] [] c.t.s.o.s.OmadaLinuxMain(): No active profile set, falling back to 1 default profile: "default"

MongoDB Logs

andrew@jane: ~
$ cat /data/omada/logs/mongod.log
2024-02-01T21:08:44.111+0000 I CONTROL  [initandlisten] MongoDB starting : pid=169 port=27217 dbpath=../data/db 64-bit host=jane
2024-02-01T21:08:44.114+0000 I CONTROL  [initandlisten] db version v3.6.8
2024-02-01T21:08:44.114+0000 I CONTROL  [initandlisten] git version: 8e540c0b6db93ce994cc548f000900bdc740f80a
2024-02-01T21:08:44.114+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020
2024-02-01T21:08:44.114+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2024-02-01T21:08:44.114+0000 I CONTROL  [initandlisten] modules: none
2024-02-01T21:08:44.115+0000 I CONTROL  [initandlisten] build environment:
2024-02-01T21:08:44.115+0000 I CONTROL  [initandlisten]     distarch: aarch64
2024-02-01T21:08:44.115+0000 I CONTROL  [initandlisten]     target_arch: aarch64
2024-02-01T21:08:44.115+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27217 }, processManagement: { pidFilePath: "../data/mongo.pid" }, storage: { dbPath: "../data/db", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "../logs/mongod.log" } }
2024-02-01T21:08:44.118+0000 I STORAGE  [initandlisten]
2024-02-01T21:08:44.118+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2024-02-01T21:08:44.118+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2024-02-01T21:08:44.119+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=256M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),cache_cursors=false,compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2024-02-01T21:08:44.341+0000 I STORAGE  [initandlisten] WiredTiger message [1706821724:341728][169:0xffffa804f040], txn-recover: Set global recovery timestamp: 0
2024-02-01T21:08:44.476+0000 I CONTROL  [initandlisten]
2024-02-01T21:08:44.476+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2024-02-01T21:08:44.476+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2024-02-01T21:08:44.477+0000 I CONTROL  [initandlisten]
2024-02-01T21:08:44.556+0000 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: 2d5e86ff-55a6-4547-baf5-961590eb25f2
2024-02-01T21:08:44.624+0000 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.6
2024-02-01T21:08:44.662+0000 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: 9d2ffe55-25d5-43e1-a9d0-e5fabb355340
2024-02-01T21:08:44.717+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '../data/db/diagnostic.data'
2024-02-01T21:08:44.722+0000 I NETWORK  [initandlisten] waiting for connections on port 27217
2024-02-01T21:08:44.722+0000 I STORAGE  [LogicalSessionCacheRefresh] createCollection: config.system.sessions with generated UUID: 428b8020-868d-4552-8aef-aa966846804f
2024-02-01T21:08:44.808+0000 I INDEX    [LogicalSessionCacheRefresh] build index on: config.system.sessions properties: { v: 2, key: { lastUse: 1 }, name: "lsidTTLIndex", ns: "config.system.sessions", expireAfterSeconds: 1800 }
2024-02-01T21:08:44.808+0000 I INDEX    [LogicalSessionCacheRefresh]     building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2024-02-01T21:08:44.820+0000 I INDEX    [LogicalSessionCacheRefresh] build index done.  scanned 0 total records. 0 secs
2024-02-01T21:09:04.973+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:38572 #1 (1 connection now open)
2024-02-01T21:09:04.975+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:38586 #2 (2 connections now open)
2024-02-01T21:09:05.354+0000 I NETWORK  [conn1] received client metadata from 127.0.0.1:38572 conn1: { driver: { name: "mongo-java-driver|sync", version: "4.6.1" }, os: { type: "Linux", name: "Linux", architecture: "aarch64", version: "6.5.0-1009-raspi" }, platform: "Java/Private Build/17.0.9+9-Ubuntu-120.04" }
2024-02-01T21:09:05.356+0000 I NETWORK  [conn2] received client metadata from 127.0.0.1:38586 conn2: { driver: { name: "mongo-java-driver|sync", version: "4.6.1" }, os: { type: "Linux", name: "Linux", architecture: "aarch64", version: "6.5.0-1009-raspi" }, platform: "Java/Private Build/17.0.9+9-Ubuntu-120.04" }
2024-02-01T21:09:06.668+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:38594 #3 (3 connections now open)
2024-02-01T21:09:06.671+0000 I NETWORK  [conn3] received client metadata from 127.0.0.1:38594 conn3: { driver: { name: "mongo-java-driver|sync", version: "4.6.1" }, os: { type: "Linux", name: "Linux", architecture: "aarch64", version: "6.5.0-1009-raspi" }, platform: "Java/Private Build/17.0.9+9-Ubuntu-120.04" }
2024-02-01T21:09:07.171+0000 I NETWORK  [conn3] end connection 127.0.0.1:38594 (2 connections now open)
2024-02-01T21:09:07.175+0000 I NETWORK  [conn1] end connection 127.0.0.1:38572 (1 connection now open)
2024-02-01T21:09:07.177+0000 I NETWORK  [conn2] end connection 127.0.0.1:38586 (0 connections now open)
2024-02-01T21:11:23.294+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 259, after asserts: 470, after backgroundFlushing: 520, after connections: 548, after dur: 548, after extra_info: 579, after globalLock: 604, after locks: 604, after logicalSessionRecordCache: 638, after network: 670, after opLatencies: 670, after opcounters: 670, after opcountersRepl: 670, after repl: 780, after security: 780, after storageEngine: 847, after tcmalloc: 927, after transactions: 927, after transportSecurity: 937, after wiredTiger: 994, at end: 1216 }
2024-02-01T21:11:29.277+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 113, after asserts: 333, after backgroundFlushing: 370, after connections: 401, after dur: 401, after extra_info: 467, after globalLock: 500, after locks: 500, after logicalSessionRecordCache: 536, after network: 602, after opLatencies: 602, after opcounters: 630, after opcountersRepl: 630, after repl: 717, after security: 717, after storageEngine: 802, after tcmalloc: 932, after transactions: 932, after transportSecurity: 969, after wiredTiger: 1104, at end: 2261 }
2024-02-01T21:11:35.869+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 319, after asserts: 531, after backgroundFlushing: 582, after connections: 610, after dur: 610, after extra_info: 715, after globalLock: 755, after locks: 845, after logicalSessionRecordCache: 918, after network: 1004, after opLatencies: 1041, after opcounters: 1041, after opcountersRepl: 1041, after repl: 1189, after security: 1189, after storageEngine: 1406, after tcmalloc: 1748, after transactions: 1852, after transportSecurity: 1904, after wiredTiger: 2368, at end: 3577 }
2024-02-01T21:11:43.987+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 396, after asserts: 827, after backgroundFlushing: 1055, after connections: 1189, after dur: 1189, after extra_info: 1242, after globalLock: 1411, after locks: 1411, after logicalSessionRecordCache: 1556, after network: 1668, after opLatencies: 1757, after opcounters: 1757, after opcountersRepl: 1757, after repl: 1846, after security: 1896, after storageEngine: 2143, after tcmalloc: 2430, after transactions: 2557, after transportSecurity: 2666, after wiredTiger: 2998, at end: 5054 }
2024-02-01T21:11:45.121+0000 I COMMAND  [PeriodicTaskRunner] task: DBConnectionPool-cleaner took: 117ms
2024-02-01T21:11:49.974+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 392, after asserts: 580, after backgroundFlushing: 676, after connections: 712, after dur: 712, after extra_info: 800, after globalLock: 893, after locks: 893, after logicalSessionRecordCache: 941, after network: 1133, after opLatencies: 1200, after opcounters: 1200, after opcountersRepl: 1200, after repl: 1262, after security: 1309, after storageEngine: 1444, after tcmalloc: 1711, after transactions: 1806, after transportSecurity: 1847, after wiredTiger: 2346, at end: 3509 }
2024-02-01T21:11:56.397+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 371, after asserts: 750, after backgroundFlushing: 895, after connections: 1027, after dur: 1027, after extra_info: 1072, after globalLock: 1130, after locks: 1130, after logicalSessionRecordCache: 1211, after network: 1249, after opLatencies: 1351, after opcounters: 1351, after opcountersRepl: 1351, after repl: 1442, after security: 1494, after storageEngine: 1630, after tcmalloc: 1842, after transactions: 1883, after transportSecurity: 1918, after wiredTiger: 2364, at end: 3118 }
2024-02-01T21:12:04.749+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 204, after asserts: 619, after backgroundFlushing: 752, after connections: 959, after dur: 959, after extra_info: 1140, after globalLock: 1196, after locks: 1258, after logicalSessionRecordCache: 1451, after network: 1721, after opLatencies: 1916, after opcounters: 1916, after opcountersRepl: 1916, after repl: 2088, after security: 2135, after storageEngine: 2209, after tcmalloc: 2625, after transactions: 2766, after transportSecurity: 2827, after wiredTiger: 3463, at end: 5234 }
2024-02-01T21:12:11.497+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 220, after asserts: 654, after backgroundFlushing: 699, after connections: 742, after dur: 742, after extra_info: 840, after globalLock: 908, after locks: 974, after logicalSessionRecordCache: 1065, after network: 1136, after opLatencies: 1189, after opcounters: 1189, after opcountersRepl: 1189, after repl: 1279, after security: 1338, after storageEngine: 1607, after tcmalloc: 1959, after transactions: 1987, after transportSecurity: 2081, after wiredTiger: 2654, at end: 3641 }
2024-02-01T21:12:20.985+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 459, after asserts: 1381, after backgroundFlushing: 1580, after connections: 1888, after dur: 1888, after extra_info: 2130, after globalLock: 2316, after locks: 2462, after logicalSessionRecordCache: 2620, after network: 2944, after opLatencies: 3026, after opcounters: 3088, after opcountersRepl: 3088, after repl: 3271, after security: 3408, after storageEngine: 3604, after tcmalloc: 3919, after transactions: 4006, after transportSecurity: 4063, after wiredTiger: 4871, at end: 6322 }
2024-02-01T21:12:30.353+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 559, after asserts: 1327, after backgroundFlushing: 1380, after connections: 1400, after dur: 1400, after extra_info: 1560, after globalLock: 1608, after locks: 1608, after logicalSessionRecordCache: 1691, after network: 1947, after opLatencies: 2033, after opcounters: 2079, after opcountersRepl: 2079, after repl: 2187, after security: 2220, after storageEngine: 2445, after tcmalloc: 2863, after transactions: 3005, after transportSecurity: 3148, after wiredTiger: 3704, at end: 5172 }
2024-02-01T21:12:37.559+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 306, after asserts: 721, after backgroundFlushing: 902, after connections: 1016, after dur: 1016, after extra_info: 1091, after globalLock: 1240, after locks: 1240, after logicalSessionRecordCache: 1353, after network: 1521, after opLatencies: 1638, after opcounters: 1753, after opcountersRepl: 1753, after repl: 1912, after security: 1951, after storageEngine: 2206, after tcmalloc: 2446, after transactions: 2446, after transportSecurity: 2508, after wiredTiger: 3048, at end: 4534 }
2024-02-01T21:12:46.155+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 218, after asserts: 502, after backgroundFlushing: 623, after connections: 623, after dur: 623, after extra_info: 670, after globalLock: 733, after locks: 782, after logicalSessionRecordCache: 848, after network: 901, after opLatencies: 948, after opcounters: 948, after opcountersRepl: 948, after repl: 1079, after security: 1160, after storageEngine: 1303, after tcmalloc: 1562, after transactions: 1660, after transportSecurity: 1698, after wiredTiger: 2250, at end: 3555 }
2024-02-01T21:12:49.469+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 74, after asserts: 218, after backgroundFlushing: 288, after connections: 339, after dur: 339, after extra_info: 339, after globalLock: 406, after locks: 518, after logicalSessionRecordCache: 533, after network: 533, after opLatencies: 533, after opcounters: 533, after opcountersRepl: 533, after repl: 693, after security: 693, after storageEngine: 760, after tcmalloc: 827, after transactions: 891, after transportSecurity: 922, after wiredTiger: 973, at end: 2199 }
2024-02-01T21:12:54.127+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 388, after asserts: 569, after backgroundFlushing: 609, after connections: 609, after dur: 609, after extra_info: 668, after globalLock: 740, after locks: 794, after logicalSessionRecordCache: 874, after network: 1052, after opLatencies: 1132, after opcounters: 1132, after opcountersRepl: 1132, after repl: 1257, after security: 1301, after storageEngine: 1391, after tcmalloc: 1565, after transactions: 1615, after transportSecurity: 1615, after wiredTiger: 1689, at end: 2611 }
2024-02-01T21:13:01.014+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 200, after asserts: 419, after backgroundFlushing: 472, after connections: 514, after dur: 514, after extra_info: 551, after globalLock: 594, after locks: 594, after logicalSessionRecordCache: 625, after network: 683, after opLatencies: 683, after opcounters: 683, after opcountersRepl: 683, after repl: 804, after security: 804, after storageEngine: 905, after tcmalloc: 1065, after transactions: 1151, after transportSecurity: 1206, after wiredTiger: 1759, at end: 4127 }
2024-02-01T21:13:07.210+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 432, after asserts: 727, after backgroundFlushing: 866, after connections: 923, after dur: 923, after extra_info: 947, after globalLock: 1030, after locks: 1060, after logicalSessionRecordCache: 1121, after network: 1250, after opLatencies: 1312, after opcounters: 1312, after opcountersRepl: 1312, after repl: 1366, after security: 1394, after storageEngine: 1514, after tcmalloc: 1775, after transactions: 1852, after transportSecurity: 1881, after wiredTiger: 2210, at end: 3289 }
2024-02-01T21:13:12.922+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 269, after asserts: 554, after backgroundFlushing: 725, after connections: 795, after dur: 795, after extra_info: 915, after globalLock: 951, after locks: 1027, after logicalSessionRecordCache: 1063, after network: 1216, after opLatencies: 1254, after opcounters: 1254, after opcountersRepl: 1254, after repl: 1364, after security: 1393, after storageEngine: 1587, after tcmalloc: 1768, after transactions: 1847, after transportSecurity: 1884, after wiredTiger: 2292, at end: 3289 }
2024-02-01T21:13:21.762+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 255, after asserts: 650, after backgroundFlushing: 805, after connections: 905, after dur: 905, after extra_info: 990, after globalLock: 1188, after locks: 1279, after logicalSessionRecordCache: 1431, after network: 1653, after opLatencies: 1791, after opcounters: 1854, after opcountersRepl: 1854, after repl: 1989, after security: 2058, after storageEngine: 2387, after tcmalloc: 2697, after transactions: 2826, after transportSecurity: 2966, after wiredTiger: 4633, at end: 6037 }
2024-02-01T21:13:29.570+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 146, after asserts: 443, after backgroundFlushing: 494, after connections: 561, after dur: 561, after extra_info: 642, after globalLock: 679, after locks: 679, after logicalSessionRecordCache: 833, after network: 1026, after opLatencies: 1071, after opcounters: 1101, after opcountersRepl: 1101, after repl: 1196, after security: 1261, after storageEngine: 1428, after tcmalloc: 1576, after transactions: 1621, after transportSecurity: 1673, after wiredTiger: 1905, at end: 3601 }
2024-02-01T21:13:38.466+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 282, after asserts: 614, after backgroundFlushing: 699, after connections: 742, after dur: 742, after extra_info: 880, after globalLock: 1010, after locks: 1113, after logicalSessionRecordCache: 1237, after network: 1280, after opLatencies: 1412, after opcounters: 1601, after opcountersRepl: 1601, after repl: 1781, after security: 1851, after storageEngine: 2146, after tcmalloc: 2416, after transactions: 2553, after transportSecurity: 2739, after wiredTiger: 3573, at end: 5230 }
2024-02-01T21:13:46.589+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 304, after asserts: 695, after backgroundFlushing: 847, after connections: 941, after dur: 941, after extra_info: 986, after globalLock: 1194, after locks: 1282, after logicalSessionRecordCache: 1438, after network: 1830, after opLatencies: 1962, after opcounters: 2069, after opcountersRepl: 2069, after repl: 2143, after security: 2199, after storageEngine: 2447, after tcmalloc: 2686, after transactions: 2740, after transportSecurity: 2784, after wiredTiger: 3202, at end: 5569 }
2024-02-01T21:13:53.064+0000 I COMMAND  [LogicalSessionCacheRefresh] command config.$cmd command: createIndexes { createIndexes: "system.sessions", indexes: [ { key: { lastUse: 1 }, name: "lsidTTLIndex", expireAfterSeconds: 1800 } ], $db: "config" } numYields:0 reslen:101 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } } } protocol:op_msg 4095ms
2024-02-01T21:13:58.742+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 395, after asserts: 659, after backgroundFlushing: 911, after connections: 1104, after dur: 1104, after extra_info: 1170, after globalLock: 1224, after locks: 1224, after logicalSessionRecordCache: 1373, after network: 2041, after opLatencies: 2273, after opcounters: 2273, after opcountersRepl: 2273, after repl: 2600, after security: 2779, after storageEngine: 3104, after tcmalloc: 3666, after transactions: 3800, after transportSecurity: 3871, after wiredTiger: 4909, at end: 6711 }
2024-02-01T21:14:03.288+0000 I WRITE    [LogicalSessionCacheRefresh] remove config.system.sessions command: { q: { _id: { id: UUID("a5270764-39d4-4666-a10e-c0dd77d24826"), uid: BinData(0, E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855) } }, limit: 0 } planSummary: IDHACK keysExamined:0 docsExamined:0 ndeleted:0 numYields:1 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 2 } } } 3190ms
2024-02-01T21:14:06.068+0000 I COMMAND  [LogicalSessionCacheRefresh] command config.$cmd command: delete { delete: "system.sessions", ordered: false, writeConcern: { w: "majority", wtimeout: 15000 }, $db: "config" } numYields:0 reslen:29 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 2 } } } protocol:op_msg 6862ms
2024-02-01T21:14:06.340+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 287, after asserts: 532, after backgroundFlushing: 633, after connections: 687, after dur: 687, after extra_info: 839, after globalLock: 839, after locks: 872, after logicalSessionRecordCache: 1056, after network: 1244, after opLatencies: 1363, after opcounters: 1383, after opcountersRepl: 1383, after repl: 1567, after security: 1612, after storageEngine: 1958, after tcmalloc: 2120, after transactions: 2244, after transportSecurity: 2301, after wiredTiger: 2874, at end: 3633 }
2024-02-01T21:14:10.242+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 110, after asserts: 135, after backgroundFlushing: 135, after connections: 135, after dur: 173, after extra_info: 287, after globalLock: 372, after locks: 462, after logicalSessionRecordCache: 519, after network: 699, after opLatencies: 740, after opcounters: 816, after opcountersRepl: 816, after repl: 972, after security: 1027, after storageEngine: 1127, after tcmalloc: 1435, after transactions: 1580, after transportSecurity: 1640, after wiredTiger: 1967, at end: 3041 }
2024-02-01T21:14:14.090+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 183, after asserts: 405, after backgroundFlushing: 462, after connections: 462, after dur: 462, after extra_info: 505, after globalLock: 549, after locks: 568, after logicalSessionRecordCache: 598, after network: 646, after opLatencies: 670, after opcounters: 670, after opcountersRepl: 670, after repl: 758, after security: 758, after storageEngine: 825, after tcmalloc: 860, after transactions: 890, after transportSecurity: 927, after wiredTiger: 1200, at end: 1696 }
2024-02-01T21:14:16.750+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 172, after asserts: 259, after backgroundFlushing: 317, after connections: 317, after dur: 317, after extra_info: 392, after globalLock: 444, after locks: 444, after logicalSessionRecordCache: 485, after network: 539, after opLatencies: 539, after opcounters: 539, after opcountersRepl: 539, after repl: 679, after security: 679, after storageEngine: 702, after tcmalloc: 779, after transactions: 779, after transportSecurity: 813, after wiredTiger: 857, at end: 1125 }
2024-02-01T21:14:21.688+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 335, after asserts: 540, after backgroundFlushing: 597, after connections: 634, after dur: 634, after extra_info: 704, after globalLock: 739, after locks: 775, after logicalSessionRecordCache: 857, after network: 985, after opLatencies: 1081, after opcounters: 1113, after opcountersRepl: 1113, after repl: 1192, after security: 1235, after storageEngine: 1296, after tcmalloc: 1418, after transactions: 1418, after transportSecurity: 1463, after wiredTiger: 1827, at end: 2533 }
2024-02-01T21:14:28.303+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 576, after asserts: 1043, after backgroundFlushing: 1154, after connections: 1218, after dur: 1218, after extra_info: 1254, after globalLock: 1331, after locks: 1331, after logicalSessionRecordCache: 1428, after network: 1537, after opLatencies: 1629, after opcounters: 1661, after opcountersRepl: 1661, after repl: 1744, after security: 1779, after storageEngine: 2010, after tcmalloc: 2233, after transactions: 2357, after transportSecurity: 2457, after wiredTiger: 2780, at end: 4736 }
2024-02-01T21:14:37.788+0000 I COMMAND  [ftdc] serverStatus was very slow: { after basic: 299, after asserts: 584, after backgroundFlushing: 643, after connections: 703, after dur: 703, after extra_info: 806, after globalLock: 1029, after locks: 1029, after logicalSessionRecordCache: 1159, after network: 1313, after opLatencies: 1487, after opcounters: 1642, after opcountersRepl: 1642, after repl: 1881, after security: 1938, after storageEngine: 2186, after tcmalloc: 2363, after transactions: 2384, after transportSecurity: 2431, after wiredTiger: 2948, at end: 4862 }
2024-02-01T21:14:46.282+0000 I COMMAND  [PeriodicTaskRunner] task: DBConnectionPool-cleaner took: 367ms
2024-02-01T21:14:53.041+0000 I COMMAND  [PeriodicTaskRunner] task: UnusedLockCleaner took: 284ms
2024-02-01T21:14:56.714+0000 I COMMAND  [PeriodicTaskRunner] task: DBConnectionPool-cleaner took: 133m

Additional Context

In case any of this is relevant:

Host Ubuntu version output:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 23.10
Release:        23.10
Codename:       mantic

Docker version output:

$ docker --version
Docker version 25.0.2, build 29cf629

And pc arch output:

$ uname -a
Linux jane 6.5.0-1009-raspi #12-Ubuntu SMP PREEMPT_DYNAMIC Wed Jan 17 11:45:08 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Some more stuff I've seen you ask for in other issues:

$ docker images --filter=reference='mbentley/omada-controller' --digests
REPOSITORY                  TAG       DIGEST                                                                    IMAGE ID       CREATED      SIZE
mbentley/omada-controller   latest    sha256:f1e50b9d45488fb97ed2d57256b1020a2107b20915b92d0c828e3b3927805910   f18c02b39de5   9 days ago   624MB

$ docker info --format '{{json .OSType}} {{json .Architecture}}'
"linux" "aarch64"

andrew@jane: ~
$ cat /etc/os-release
docker info
dpkg --print-architecture
for i in /etc/apt/sources.list /etc/apt/sources.list.d/*; do echo "### $i"; cat "$i";echo;done
PRETTY_NAME="Ubuntu 23.10"
NAME="Ubuntu"
VERSION_ID="23.10"
VERSION="23.10 (Mantic Minotaur)"
VERSION_CODENAME=mantic
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=mantic
LOGO=ubuntu-logo
Client: Docker Engine - Community
 Version:    25.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 3
 Server Version: 25.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-1009-raspi
 Operating System: Ubuntu 23.10
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 841.3MiB
 Name: jane
 ID: 8d2b642e-4781-4eb8-846d-62ba05585974
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

arm64
### /etc/apt/sources.list
## Note, this file is written by cloud-init on first boot of an instance
## modifications made here will not survive a re-bundle.
## if you wish to make changes you can:
## a.) add 'apt_preserve_sources_list: true' to /etc/cloud/cloud.cfg
##     or do the same in user-data
## b.) add sources in /etc/apt/sources.list.d
## c.) make changes to template file /etc/cloud/templates/sources.list.tmpl

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://ports.ubuntu.com/ubuntu-ports mantic main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://ports.ubuntu.com/ubuntu-ports mantic-updates main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://ports.ubuntu.com/ubuntu-ports mantic universe
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic universe
deb http://ports.ubuntu.com/ubuntu-ports mantic-updates universe
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://ports.ubuntu.com/ubuntu-ports mantic multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic multiverse
deb http://ports.ubuntu.com/ubuntu-ports mantic-updates multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://ports.ubuntu.com/ubuntu-ports mantic-backports main restricted universe multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-backports main restricted universe multiverse

deb http://ports.ubuntu.com/ubuntu-ports mantic-security main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-security main restricted
deb http://ports.ubuntu.com/ubuntu-ports mantic-security universe
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-security universe
deb http://ports.ubuntu.com/ubuntu-ports mantic-security multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-security multiverse

### /etc/apt/sources.list.d/docker.list
deb [arch=arm64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu   mantic stable

### /etc/apt/sources.list.d/mongodb-org-3.6.list
deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/3.6 multiverse


andrew@jane: ~
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 23.10"
NAME="Ubuntu"
VERSION_ID="23.10"
VERSION="23.10 (Mantic Minotaur)"
VERSION_CODENAME=mantic
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=mantic
LOGO=ubuntu-logo

andrew@jane: ~
$ docker info
Client: Docker Engine - Community
 Version:    25.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 3
 Server Version: 25.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-1009-raspi
 Operating System: Ubuntu 23.10
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 841.3MiB
 Name: jane
 ID: 8d2b642e-4781-4eb8-846d-62ba05585974
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false


andrew@jane: ~
$ dpkg --print-architecture
arm64

andrew@jane: ~
$ for i in /etc/apt/sources.list /etc/apt/sources.list.d/*; do echo "### $i"; cat "$i";echo;done
### /etc/apt/sources.list
## Note, this file is written by cloud-init on first boot of an instance
## modifications made here will not survive a re-bundle.
## if you wish to make changes you can:
## a.) add 'apt_preserve_sources_list: true' to /etc/cloud/cloud.cfg
##     or do the same in user-data
## b.) add sources in /etc/apt/sources.list.d
## c.) make changes to template file /etc/cloud/templates/sources.list.tmpl

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://ports.ubuntu.com/ubuntu-ports mantic main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://ports.ubuntu.com/ubuntu-ports mantic-updates main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://ports.ubuntu.com/ubuntu-ports mantic universe
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic universe
deb http://ports.ubuntu.com/ubuntu-ports mantic-updates universe
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://ports.ubuntu.com/ubuntu-ports mantic multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic multiverse
deb http://ports.ubuntu.com/ubuntu-ports mantic-updates multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://ports.ubuntu.com/ubuntu-ports mantic-backports main restricted universe multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-backports main restricted universe multiverse

deb http://ports.ubuntu.com/ubuntu-ports mantic-security main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-security main restricted
deb http://ports.ubuntu.com/ubuntu-ports mantic-security universe
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-security universe
deb http://ports.ubuntu.com/ubuntu-ports mantic-security multiverse
# deb-src http://ports.ubuntu.com/ubuntu-ports mantic-security multiverse

### /etc/apt/sources.list.d/docker.list
deb [arch=arm64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu   mantic stable

### /etc/apt/sources.list.d/mongodb-org-3.6.list
deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/3.6 multiverse

[mbentley]

Honestly, your system is may be swapping like mad with Total Memory: 841.3MiB. That's not a lot of RAM at all in order to run the controller. It may also be possible that your sd card is really slow for read/writes, assuming it is using an sd card. The log messages from MongoDB are concerning.

You can check with another terminal open, regularly checking free -m and docker stats when the container is launched to see what is happening with your system's resources. Looking at htop is good too, assuming you know what you're looking at of course. Not always intuitive if you haven't used it previously.

Just to make sure you're definitely doing this, I would try a few different CMDs to override the default amount of memory:

512m:
/usr/bin/java -server -Xms128m -Xmx512m -XX:MaxHeapFreeRatio=60 -XX:MinHeapFreeRatio=30 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tplink/EAPController/logs/java_heapdump.hprof -Djava.awt.headless=true --add-opens java.base/java.util=ALL-UNNAMED -cp /opt/tplink/EAPController/lib/*::/opt/tplink/EAPController/properties: com.tplink.smb.omada.starter.OmadaLinuxMain

Or even 256m:
/usr/bin/java -server -Xms128m -Xmx256m -XX:MaxHeapFreeRatio=60 -XX:MinHeapFreeRatio=30 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tplink/EAPController/logs/java_heapdump.hprof -Djava.awt.headless=true --add-opens java.base/java.util=ALL-UNNAMED -cp /opt/tplink/EAPController/lib/*::/opt/tplink/EAPController/properties: com.tplink.smb.omada.starter.OmadaLinuxMain

The biggest issue I would think it going on is that MongoDB is gonna eat up a good bit of RAM too.