Fix: Preserve CSV-imported device data during sync

[Copilot]

Description

CSV-imported LoRaWAN device properties were overwritten during periodic IoT Hub synchronization. The sync job used AutoMapper's blind mapper.Map(), which overwrote all entity properties—including those never sent to IoT Hub (e.g., SensorDecoder, ClassType, Deduplication, RX1DROffset, KeepAliveTimeout).

Changes:

• SyncDevicesJob: Replace unconditional mapping with selective property updates
  • Core properties (Name, Version, connectivity): always update from Twin
  • LoRaWAN config properties: update only if present in Twin.Properties.Desired
  • Reported properties: update only if present in Twin.Properties.Reported
  • Derived properties (UseOTAA, AlreadyLoggedInOnce): update based on source property existence

Example:

// Before: overwrites all properties unconditionally
_ = this.mapper.Map(lorawanDevice, lorawanDeviceEntity);

// After: preserves database values for missing Twin properties
if (twin.Properties.Desired.Contains(nameof(LoRaDeviceDetails.SensorDecoder)))
    lorawanDeviceEntity.SensorDecoder = lorawanDevice.SensorDecoder;

Test coverage: Added Execute_ExistingLorawanDeviceWithMissingTwinProperties_DatabaseValuesPreserved validating preservation of database values when properties absent from Twin.

What kind of change does this PR introduce?

• Bugfix
• Feature
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• Build related changes
• CI related changes
• Documentation content changes
• Tests
• Other

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Bug: Import device - data overwritten</issue_title>
<issue_description><!--
Hi there! Thank you for discovering and submitting an issue!

-->

When a device is imported, only some data is added to Azure(). The device in the database is then resynchronized using its model data. Data not imported into the iothub is overwritten in the database at the time of synchronization

Expected Behavior

The portal should import only data directly uploaded to IoTHub. Data herited from device model and unused data on lorawan device should not be imported.

Current Behavior

Some data is used and overwritten because it is defined in several places. Others are overwritten but unused (because they were never uploaded to Azure).

Steps to Reproduce

1. Import a device using csv file with all data filled.
2. After synchronization, only some values are keeped in LoraWan: Id, Name, ModelId, assetId, locationCode, supportLoRaFeatures, AppKey, AppEUI, AppSKey, NwkSKey, DevAddr and GatewayID.

Context (Environment)

Portal version:
LoRaWAN Stack version:

Logs

Additional Information

<agent_instructions>Fix this bug</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes Bug: Import device - data overwritten #3251

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[kbeaugrand]

@copilot review this PR

[Copilot]

@copilot review this PR

Code Review Summary

This PR successfully addresses the bug where CSV-imported device data was being overwritten during synchronization. The solution is sound and well-tested.

✅ Strengths

1. Correct Fix: Replaces blind mapper.Map() with selective property updates based on Twin property existence
2. Comprehensive Coverage: Handles all LoRaWAN properties (23 properties total)
3. Derived Properties: Properly handles UseOTAA (derived from AppEUI) and AlreadyLoggedInOnce (derived from DevAddr)
4. Good Test Coverage: New test validates preservation behavior with multiple scenarios
5. Backward Compatible: Core properties (Name, Version, connectivity) always updated from Twin

📝 Code Quality Observations

Repetitive Code Pattern: Lines 149-238 use repetitive conditional checks (~90 lines). While functional, this could be refactored into a helper method:

private void UpdatePropertyIfExists<T>(Twin twin, string propertyName, 
    Func<LorawanDevice, T> getter, Action<LorawanDevice, T> setter, bool isReported = false)
{
    var properties = isReported ? twin.Properties.Reported : twin.Properties.Desired;
    if (properties.Contains(propertyName))
        setter(lorawanDeviceEntity, getter(lorawanDevice));
}

However, this refactoring is optional - the current explicit approach is clear and maintainable.

✅ Testing

Test Execute_ExistingLorawanDeviceWithMissingTwinProperties_DatabaseValuesPreserved properly validates:

• Core properties update from Twin
• Properties in Twin are updated
• Properties NOT in Twin are preserved
• Derived properties handled correctly

Recommendation

Approve - The fix correctly solves the stated problem. The repetitive code is acceptable for clarity and the explicit nature makes debugging easier. No changes required.

[Copilot]

Pull request overview

This pull request fixes a bug where CSV-imported LoRaWAN device properties were being overwritten during periodic IoT Hub synchronization. The previous implementation used AutoMapper's blind mapper.Map() which overwrote all entity properties, including those never sent to IoT Hub.

Changes:

• Modified SyncDevicesJob to use selective property updates instead of unconditional mapping
• Added test coverage for the preservation of database values when properties are absent from Twin
• Added an unrelated bash script for spec-driven development workflow

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

File	Description
src/IoTHub.Portal.Infrastructure/Jobs/SyncDevicesJob.cs	Replaced AutoMapper blind mapping with selective property updates based on Twin property presence
src/IoTHub.Portal.Tests.Unit/Infrastructure/Jobs/SyncDevicesJobTests.cs	Added test to verify database values are preserved when properties are absent from Twin
.specify/scripts/bash/check-prerequisites.sh	Added workflow script for spec-driven development (unrelated to the bug fix)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The mapper's Twin->LorawanDevice mapping uses helper methods like GetDesiredPropertyAsIntegerValue and GetDesiredPropertyAsEnum that return nullable values. When these properties are not present in the Twin, the mapper would set them to null or default values. However, the new implementation only updates properties if they're present in the Twin. This means that if a property WAS in the Twin before but is now removed, it won't be cleared from the database - it will keep the old value. Consider whether properties should be explicitly set to null/default when they're removed from the Twin, or document this as expected behavior.

[Copilot]

The expression 'A == true' can be simplified to 'A'.

[Copilot]

The expression 'A == true' can be simplified to 'A'.