Conversation
|
Will require a formula for Companion to validate if the configuration will/should run on the radio. Use in yaml encode and as a running check whilst editing a model to stop adding more configuration. |
Yes, but realistically only for BW radio. Given the amount of ram on a color radio, you will never be limited by it |
Ok so I still need a formula for B&W. Never say never as it has a habit of coming back to bite you... |
True. In the current thinking, there should be a MODEL_ARENA_SIZE that is set, likely per radio type or technology, will act as a limit on the radio and can be used for companion too |
We should be able to compute that from the arena size. I'll try to come up with something. |
radio/src/DYNAMIC_ARRAYS_PLAN.md
Outdated
| ``` | ||
|
|
||
| ### Companion (companion/src/firmwares/) | ||
| Companion has its own `ModelData` class with `CPN_MAX_MIXERS` etc. **No urgent change needed** -- YAML is the interface. Long-term, Companion could switch to `QVector<MixData>` for truly unlimited storage. |
There was a problem hiding this comment.
FYI QVector is now an alias for QList
and Mixes will be the first hard limit in Companion to be exceeded based on the push |
No worry, I'll try to tackle this in time. But first I need something working in simu/firmware. Companion comes next. |
|
For B&W radios this currently increases base RAM usage by ~1k for radios without RTC backup and ~2k for radios with RTC backup. This is primarily due to the increased size for arrays like 'act', 'mixState', 'activeMixes' and 'ramBackupUncompressed'. While the goal is understandable, decreasing available Lua script RAM for F4 B&W radios is not going to be popular. |
785aaf4 to
89084bb
Compare
|
Use this for SourceRef and SwitchRef to simplify get/set logic and get rid of all memcpy calls. struct SourceRef {
union {
struct {
uint8_t type; // SourceType
uint8_t flags; // SourceFlags
uint16_t index; // Index within the type (0-based)
};
uint32_t raw;
}; |
|
@raphaelcoeffic is it you or Claude borrowing source and switch refs from Companion and my yet to be officially revealed value/source branch which has been problematic on the UI widgets front which has held it up? |
3fc3f22 to
c01b24c
Compare
Working on it, it's not finished yet ;-) |
that was my suggestion. It's basically the only way to do this properly. |
c01b24c to
3752f6b
Compare
It won't be true for the majority of BW users. The allocation of mixer, .. will be dynamic, so you will use less memory when you have say only 5 or 6 mixer lines. That benefit should offset the base cost of the system, and users end up having more ram for LUA on simple models |
|
25ed372 to
a8def9e
Compare
df7aaa2 to
649fddc
Compare
That does not make sense. Mix lines are allocated sequentially in the buffer and each mix line already contains the output channel number. So even if there are gaps in the mix numbers there are no gaps in the mix lines buffer. Mixes can have multiple mix lines so there is not a 1:1 relationship - how can you specify the mix line in the output channel? |
|
Also, for the record, I am strongly opposed to adding UI changes for input/mix creation into this PR. It is already going to be very hard to review and test without having changes that are unrelated to the core goal of the PR to deal with. |
Indeed. I will refrain from any UI changes. |
c7b067c to
42e760d
Compare
fb47774 to
2bb97f1
Compare
@wimalopaan you can follow the discussion in #7219 which goes more into details. |
Replace individual get_ptr/ensure_capacity function pointers in the _extern_array union member with a single const driver pointer. This shrinks the union member from 3 pointers to 2 while allowing future callbacks without growing YamlNode. Wire up gvar_is_active via the driver so gvarValues with default sentinel (GVAR_MAX+1) or zero are skipped during YAML output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add getCurveCount(), getCurvePoints(), curveAllocAt() to properly allocate both ARENA_CURVES and ARENA_POINTS when creating curves - Bounds-check getCurvePoints(), loadCurves(), isCurveUsed() against arena section count to prevent reading adjacent section data - Fix colorlcd and B&W UI to use curveAllocAt() + storageDirty() when creating new curves - B&W UI: show only used curves + one empty slot instead of MAX_CURVES - Wire up fmd_is_active, cfn_is_active, isAlwaysActive for curves in ExternArrayDriver instances - Fix curveedit smooth toggle missing SET_DIRTY() - Add curve round-trip verification to YamlRoundTrip test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use indexed CUST_EXTERN_ARRAY for curves (matching main) so sparse curve slots preserve their index in YAML - Add curve_is_active callback using a bitmap populated by loadCurves() for O(1) checks instead of scanning all points on every save - Add Arena::trimSectionTo() for exact section count trimming - Add curveTrimTrailing() to reclaim both ARENA_CURVES and ARENA_POINTS when trailing curve slots are cleared - Color UI: call curveTrimTrailing() on curve clear - B&W UI: call curveTrimTrailing() on EXIT, setCurveUsed() on enter - Fix curveedit smooth toggle missing SET_DIRTY() - Add unit tests for trimSectionTo and curveTrimTrailing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Happy to do some testing on templates to check for impacts, if a binary for TX16S Mk1 or Mk3 can be made available. |
Cache section base pointers and loop counts before tight loops instead of calling arena accessors on every iteration. This avoids repeated indirection through g_modelArena._base + _offsets[] on each step. Loop over actual arena counts (getMixCount, getExpoCount, getLswCount) instead of MAX_* constants to skip empty slots. Remove redundant memset of static dummy in lswAddress. Cache lswAddress result in evalLogicalSwitches instead of calling it 3 times. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Avoid relying on static init ordering.
curveIsEmpty only checked the CurveHeader for zeros, but a default STANDARD curve with 5 points has an all-zero header. The UI marks curves via setCurveUsed(), so curveIsEmpty must also consult the flag bitmap to avoid trimming curves that have data. Fix tests to match UI behaviour: use lswAllocAt instead of lswAddress for out-of-range writes, and call setCurveUsed after curveAllocAt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Binaries are available for each build on the tab "Checks". Select "Run tests and package firmware", then scroll down to "Artifacts". There you can download the binaries. The following PR is probably better for perf tests since it has a few optimisations: #7219 |
|
But Mike, please test on a copy of your models, everything can, and likely will, happen, even if we conducted some testing already |
No problem. I plan to use a script to visualise behaviour, for convenience. Just some queries:
|
|
Having realised the X9D+ is not supported (doh!), I'm using GX12 as target for testing. However I am having a problem with corruption. Model setups in ETX 2.10 format are converted in Companion 3.0 apparently successfully. However after uploading to the TX they are corrupted, and mixers for CH5 and onwards are missing. Firmware: pre-3.0.0-PR7219 |
FM transitions tests would be welcome on #7219.
Check the description in #7219.
It's not supported anymore in pre-3.0. 2.12 is last version to support STM32F2 targets. Better would be to use either X9E (STM32F4) if you need the screen format (212x64), or any other STM32F4 based radio (monochrome or color).
No problem with me. |
Which model files exactly? Edit: after checking with this template I found a couple bugs when loading the models with RM Boxer (simu). It seems loading just fine now. |
Arena::ensureSectionCapacity() can reallocate the buffer, but YamlTreeWalker held stale data_override pointers into the old buffer. Refresh data_override from get_ptr() after ensure_capacity in both toNextElmt() and the IDX handler. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Legacy GV6 format is 1-based; convert to 0-based index (GV6 → index 5) - Negative GVars (-GV6) now encode inversion via negative value in ValueOrSource (value = -(index+1)), decoded by toSourceRef() - setSource() preserves GVar inversion flag as negative value - New format !gv(5) round-trips correctly through setSource/toSourceRef Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
drawValueOrSource and editValueOrSource fell through to drawSource for GVar sources, which used getSourceString (! prefix for inversion). Use drawGVarName instead, matching main branch behavior (- prefix). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
9110bff to
9792c31
Compare
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Attached are three models converted to GX12 3.0 format using Companion 3.0.0--main, Mar 18 2026, Commit 0731d5f On all three models, the mixers are visible in the Companion sim. However following transfer to the tx most mixers are missing and/or corrupted. The tx is running EdgeTX pre-3.0.0-PR7219 Model file: |
@RC-SOAR I guess you have not retried since my latest fixes this morning? I‘ve found a few things. I can however retry with these models. Also, the file you uploaded seems to be only 22 bytes big. Is that the right file? Also, I would prefer to have the original files for ETX 2.10, that would exclude potential issues with Companion conversions. since this was about testing #7219, please comment there, otherwise it is hard to follow. |
|
In a rush just now, but here's the corrected zip file containing the .etx file with three models after conversion from TX16S 2.10 to GX12 3.0 format using Companion 3.0.0--main, Mar 18 2026, Commit 0731d5f Mixers appear in the Companion sim but are missing and/or corrupted on the GX12 running EdgeTX pre-3.0.0-PR7219 I'll do some tests with the latest ETX build when I get back later. |
Thx, these models do load with the GX12 here and latest from this PR as well as #7219 |
Replace the fixed char inputNames[MAX_INPUTS][LEN_INPUT_NAME] array in ModelData with a sparse arena section (ARENA_INPUT_NAMES) indexed by a uint8_t inputNameIndex[MAX_INPUTS] side-array. Only named inputs consume arena space; 0xFF in the index means "no name". Accessor API: inputName() for O(1) read, inputNameAlloc() for alloc-on-demand, inputNameClear() with swap-with-last compaction. YAML serialization uses YAML_IDX_CUST so the custom index read/write maps between sparse arena slots and YAML input numbers directly — no expand/compact buffers needed. Saves 96 bytes (color) / 64 bytes (B&W) from ModelData for typical models that name only a few inputs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2d2774d to
980e6fd
Compare

Fixes #7155
Summary
Replace
ModelData's fixed-size arrays and bit-packed cross-references with arena-based dynamic allocation and 32-bit structured source/switch types. This shrinksModelDatafrom ~6.5 KB to ~2.5 KB, fixes the H7 telemetry sensor overflow bug, and removes all source/switch index limits.Full technical details in
radio/src/DYNAMIC_ARRAYS_PLAN.md.Phase 1: Accessor Abstraction
All
ModelDataarray access routed through accessor functions (mixAddress(),expoAddress(),lswAddress(), etc.). No directg_model.mixData[i]remains in non-test code.Phase 2: Arena Allocator
Six arrays (mixes, expos, curves, points, logical switches, custom functions) moved from
ModelDatato a sharedModelArenawith section-based layout and YAMLYDT_EXTERN_ARRAYsupport.Phase 3b: Structured Source/Switch References
Replaced the monolithic
MixSources/SwitchSourcesenums and 10-bit packed fields with three 32-bit structured types:SourceRef—{type, flags, index}replacessrcRaw:10SwitchRef—{type, flags, index}replacesswtch:10ValueOrSource—{value, isSource, srcType}replacesSourceNumVal11-bit packedAll structs, GUI editors (colorlcd + 212x64 + 128x64), YAML serialization, and the Lua API have been fully migrated.
Phase 4: Dynamic Arena Sizing
Arena dynamically sized to actual model usage after load. Sections grow on demand via
ensureSectionCapacity(). GUI capacity checks usefreeBytes().trimTrailingEmpty()reclaims sparse indexed slots.Phase 6: Growable Arena
Arena buffer dynamically growable using a platform-split strategy:
_capacitywithin the pre-reserved buffer — no allocation, no copy, no pointer invalidation.malloc(). Start at 1 KB, grow via allocate-copy-free.shrinkToFit()only reallocates when the buffer grew beyond the initial size.Generic Arena + Radio Arena
Arena class refactored to use a const
ArenaDescdescriptor (numSections + elemSizes) — all methods compiled once, no template duplication. Radio custom functions moved from fixedRadioData::customFn[64]to a separateg_radioArena(1 section, 256 B initial, 1 KB max), saving ~1 KB in RadioData.evalFunctions()takes an explicit count parameter — the pointer-identity hack is removed.Phase 5a: Eliminate GV_RANGE encoding from LimitData
Replaced the old
GV_RANGEsentinel-range encoding (which stole values from the edges of the 11-bit range and depended onMAX_GVARS) with a clean bit-15 flag scheme:LimitData
min/max/offsetwidened from 11-bit bitfields to fullint16_t,ppmCenterfrom 10 to 12 bits (+2 bytes per LimitData, +64 bytes per model).GV_ENCODE/GV_DECODEhelpers added at all numeric read/write boundaries. YAML format unchanged —in_read_weight/in_write_weighthandle the new encoding transparently.Phase 5b: Move FlightModes + GVars into arena
FlightModeData and GVarData moved to arena sections. GVar values extracted from FlightModeData into a separate arena section. Runtime loops and GUI updated to use runtime FM/GVar counts.
Memory Impact (Release builds)
X9D+2019 (STM32F407, tightest target):
TX16S (STM32F429/SDRAM):
X9D+2019 saves nearly 7 KB RAM and 1.9 KB FLASH. TX16S now also saves RAM thanks to the lswFm flattening.
Remaining Work
Move FlightModes + GVars into arenaDoneTest plan
🤖 Generated with Claude Code