refactor: Deserialise separate mapper config file using existing logic for tedge.toml #3889

jarhodes314 · 2025-12-08T16:46:05Z

Proposed changes

Use the existing TEdgeConfigC8yDto (etc.) structs to deserialise separate mapper config files, with a view to supporting the tedge config CLI fully.

Still TODO

Environment variable support
Full tedge config support
A tool to perform the initial migration from tedge.toml to separate mapper config
ensure we handle default first cloud connections sensibly and can set things to layer default to separate config files

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
Documentation Update (if none of the other choices apply)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist

I have read the CONTRIBUTING doc
I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
I ran just format as mentioned in CODING_GUIDELINES
I used just check as mentioned in CODING_GUIDELINES
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

Further comments

…c for tedge.toml Signed-off-by: James Rhodes <[email protected]>

jarhodes314 · 2025-12-08T16:51:03Z

crates/core/tedge/src/cli/config/commands/add.rs


    async fn execute(&self, tedge_config: TEdgeConfig) -> Result<(), MaybeFancy<anyhow::Error>> {
-        restrict_cloud_config_update("add", &self.key, &tedge_config).await?;
+        restrict_cloud_config_access("add", &self.key, &tedge_config).await?;


At the moment, all the tedge config commands are still as before, however I intend to fix that (it's just slightly tricky figuring out exactly how to achieve the fine details, but the broad "generate a TEdgeConfigDto" for the mapper config aspect is working and in use for deserialisation).

codecov · 2025-12-08T16:59:33Z

Codecov Report

❌ Patch coverage is 87.82609% with 28 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...g/src/tedge_toml/tedge_config/mapper_config/mod.rs	86.50%	16 Missing and 1 partial ⚠️
crates/core/tedge/src/cli/connect/command.rs	44.44%	5 Missing ⚠️
crates/core/tedge_mapper/src/aws/mapper.rs	0.00%	2 Missing ⚠️
crates/core/tedge_mapper/src/az/mapper.rs	0.00%	2 Missing ⚠️
...rc/tedge_toml/tedge_config/mapper_config/compat.rs	98.61%	0 Missing and 1 partial ⚠️
crates/core/tedge/src/cli/config/commands/get.rs	0.00%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

didier-wenzek · 2025-12-09T09:06:40Z

crates/common/tedge_config/src/tedge_toml/tedge_config/mapper_config/compat.rs

+    match key.split_once(".") {
+        Some(("c8y", rest)) => Some((CloudType::C8y, extract_profile_name(rest))),
+        Some(("az", rest)) => Some((CloudType::Az, extract_profile_name(rest))),
+        Some(("aws", rest)) => Some((CloudType::Aws, extract_profile_name(rest))),
+        _ => None,
+    }


It's a nice idea to move this logic out of restrict_cloud_config_acces. We have now two independent steps clearly separated.

github-actions · 2025-12-09T09:32:22Z

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
748	1	3	749	99.87	2h26m43.139356s

Failed Tests

Name	Message	⏱️ Duration	Suite
Tedge has completions for cloud profile names	!= aws azure other test	10.962 s	`Tedge Completions`

albinsuresh · 2025-12-10T10:43:26Z

crates/common/tedge_config/src/tedge_toml/tedge_config/mapper_config/mod.rs

-    + Send
-    + Sync
-    + 'static
+    Sized + ExpectedCloudType + FromCloudConfig + Send + Sync + 'static


Suggested change

Sized + ExpectedCloudType + FromCloudConfig + Send + Sync + 'static

Sized + ExpectedCloudType + FromCloudConfig + Send + Sync + std::fmt::Debug + 'static

So that the MapperConfig can also be Debug.

albinsuresh · 2025-12-10T10:55:45Z

crates/common/tedge_config/src/tedge_toml/tedge_config/mapper_config/mod.rs

-
-#[derive(Debug, Document)]
-pub struct MapperMapperConfig<M> {
+pub struct MapperMapperConfig {


Suggested change

pub struct MapperMapperConfig {

pub struct CommonMapperConfig {

albinsuresh · 2025-12-10T11:12:29Z

crates/common/tedge_config/src/tedge_toml/tedge_config/mapper_config/mod.rs


-    pub include: BridgeIncludeConfig,
+pub struct C8yBridgeConfig {
+    pub include: C8yBridgeIncludeConfig,


This could have been left as a generic mapper config, no? As it is about the local bridge connection which is valid for any mapper, whether they use it or not?

albinsuresh · 2025-12-10T11:17:03Z

crates/common/tedge_config/src/tedge_toml/tedge_config/mapper_config/mod.rs

 /// Cumulocity-specific mapper configuration fields
-#[derive(Debug, Deserialize, Document)]
-#[serde(default)]
 pub struct C8yMapperSpecificConfig {


The vision is to remove the duplication between this struct and the TEdgeConfigDtoC8y generated by the define_tedge_config! macro in a subsequent step, right? Or this is the end?

Yes, this should be the case. I think my ideal vision at this stage is for the shared configurations to be defined as a single mapper group in the define_tedge_config! macro to avoid duplication, and that will define the separate cloud configurations as well as the generalised mapper config that can be used internally to remove duplication in the various mappers.

At the moment, I think I am tending towards a solution where TEdgeConfig is proactively populated with the mapper configuration tomls where appropriate, as opposed to reading them on demand, as that makes implementing tedge config commands far simpler, and reduces the amount of code split we need between the handling the two ways of configuring mappers.

Signed-off-by: James Rhodes <[email protected]>

didier-wenzek

This work is making progress in the correct direction.

tedge config get c8y.url is correctly reading /etc/tedge/mappers/c8y.toml
tedge config get c8y.url --profile foo is correctly reading /etc/tedge/mappers/c8y.d/foo.toml
tedge config list url is correctly listing c8y settings in /etc/tedge/mappers/c8y.toml and /etc/tedge/mappers/c8y.d/foo.toml, ignoring those incorrectly added to /etc/tedge/tedge.toml
tedge connect c8y --test and sudo tedge connect c8y are working as expected.

minor:

tedge connect c8y --test and sudo tedge connect c8y complain that "Both /etc/tedge/mappers/c8y.toml and tedge.toml [c8y] exist.", even when there is no [c8y] section in /etc/tedge/tedge.toml

refactor: Deserialise separate mapper config file using existing logi…

ac3ece2

…c for tedge.toml Signed-off-by: James Rhodes <[email protected]>

jarhodes314 requested review from albinsuresh, didier-wenzek and rina23q as code owners December 8, 2025 16:46

jarhodes314 temporarily deployed to Test Pull Request December 8, 2025 16:46 — with GitHub Actions Inactive

jarhodes314 commented Dec 8, 2025

View reviewed changes

didier-wenzek reviewed Dec 9, 2025

View reviewed changes

jarhodes314 had a problem deploying to Test Auto December 9, 2025 09:11 — with GitHub Actions Failure

albinsuresh reviewed Dec 10, 2025

View reviewed changes

PoC: get tedge config get working with new mapper config

7e3219d

Signed-off-by: James Rhodes <[email protected]>

jarhodes314 temporarily deployed to Test Pull Request December 10, 2025 15:17 — with GitHub Actions Inactive

jarhodes314 had a problem deploying to Test Auto December 10, 2025 15:49 — with GitHub Actions Failure

didier-wenzek reviewed Dec 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor: Deserialise separate mapper config file using existing logic for tedge.toml #3889

refactor: Deserialise separate mapper config file using existing logic for tedge.toml #3889

jarhodes314 commented Dec 8, 2025 •

edited

Loading

Uh oh!

jarhodes314 Dec 8, 2025

Uh oh!

codecov bot commented Dec 8, 2025

Uh oh!

didier-wenzek Dec 9, 2025

Uh oh!

github-actions bot commented Dec 9, 2025 •

edited

Loading

Uh oh!

albinsuresh Dec 10, 2025

Uh oh!

albinsuresh Dec 10, 2025

Uh oh!

albinsuresh Dec 10, 2025

Uh oh!

albinsuresh Dec 10, 2025 •

edited

Loading

Uh oh!

jarhodes314 Dec 11, 2025

Uh oh!

didier-wenzek left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	Sized + ExpectedCloudType + FromCloudConfig + Send + Sync + 'static
	Sized + ExpectedCloudType + FromCloudConfig + Send + Sync + std::fmt::Debug + 'static

	pub struct MapperMapperConfig {
	pub struct CommonMapperConfig {

refactor: Deserialise separate mapper config file using existing logic for tedge.toml #3889

Are you sure you want to change the base?

refactor: Deserialise separate mapper config file using existing logic for tedge.toml #3889

Conversation

jarhodes314 commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Still TODO

Types of changes

Paste Link to the issue

Checklist

Further comments

Uh oh!

jarhodes314 Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Dec 8, 2025

Codecov Report

Uh oh!

didier-wenzek Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Robot Results

Failed Tests

Uh oh!

albinsuresh Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

albinsuresh Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

albinsuresh Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

albinsuresh Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jarhodes314 Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

didier-wenzek left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jarhodes314 commented Dec 8, 2025 •

edited

Loading

github-actions bot commented Dec 9, 2025 •

edited

Loading

albinsuresh Dec 10, 2025 •

edited

Loading