Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Sep 22, 2025

Moving server configuration settings from server_config.h to SD card for field updating capability.

Plan to Move Server Configuration to SD Card

  • Analyze current implementation in client code (loadSDCardConfiguration function)
  • Identify all configuration values in server_config.h that need to be moved to SD card
  • Create server configuration variables to replace compile-time defines
  • Implement loadServerConfigurationFromSD() function based on client pattern
  • Update server code to call SD config loading in setup()
  • Create default configuration fallbacks for essential settings
  • Create example server_config.txt file for SD card
  • Update server_config.h to only contain essential compile-time constants
  • Test configuration loading functionality
  • Update documentation with SD card configuration instructions
  • Remove fallback behavior - require SD card configuration (server)
  • Apply same changes to client - remove fallback behavior
  • Add validation for maxReportsInMemory bounds checking

Changes Made

Core Implementation ✅

  • Server: Added 23+ server configuration variables that are loaded from SD card at startup
  • Client: Updated 20+ client configuration variables to remove compile-time default dependencies
  • Implemented strict SD card configuration loading functions that halt execution if config unavailable
  • Updated all code references to use runtime configuration variables instead of compile-time defines
  • Configuration is loaded early in setup() after SD card initialization but before network setup
  • NEW: Added bounds checking for maxReportsInMemory to prevent buffer overflow

Configuration Migration ✅

Server - Moved to SD card (server_config.txt) - NOW REQUIRED:

  • Network settings: HOLOGRAM_DEVICE_KEY, static IP, MAC address, check intervals
  • Email settings: recipients, timing, Hologram API preferences
  • Server identification: SERVER_NAME, SERVER_LOCATION
  • System settings: debug mode, web refresh intervals, memory limits, log retention
  • Monthly report settings: enabled/disabled, timing

Client - Moved to SD card (tank_config.txt) - NOW REQUIRED:

  • Tank configuration: site name, tank number, height, alarm thresholds
  • Network settings: HOLOGRAM_DEVICE_KEY, phone numbers, APN settings
  • Timing configuration: sleep intervals, report timing, daily report time
  • Power management: sleep modes, wake settings
  • Logging configuration: file names, retry attempts

Kept in config files (essential compile-time constants only):

  • Hardware specifics: pin definitions, sensor types, I2C addresses
  • System buffer sizes: calibration points, timeouts
  • REMOVED: All fallback default values - SD card config is now required

Field Update Capability ✅

  • Server: Settings updated by swapping SD cards with server_config.txt
  • Client: Settings updated by swapping SD cards with tank_config.txt
  • No recompilation or code upload needed for configuration changes
  • BREAKING CHANGE: No fallback to compile-time defaults - SD card config is mandatory
  • Both server and client will halt with clear error messages if SD card or config file missing
  • Configuration validation ensures critical settings are properly configured

Required Configuration ✅

Both server and client now assume SD card configuration will always be available and will:

  • HALT if SD card cannot be initialized
  • HALT if configuration file is missing (server_config.txt or tank_config.txt)
  • HALT if critical settings like HOLOGRAM_DEVICE_KEY are not configured
  • HALT if critical phone numbers are not configured (client only)
  • Display clear error messages and instructions for resolution
  • Require operator intervention to fix configuration issues

Safety Validation ✅

  • Added bounds checking for maxReportsInMemory to prevent buffer overflow
  • If configured value exceeds MAX_TANK_REPORTS (100), it will be automatically capped
  • Warning message displayed when value is capped
  • Documentation updated to reflect the maximum limit

Testing ✅

  • Created and tested configuration parser logic with C++ mock
  • Verified key=value parsing, comment handling, data type conversion
  • Tested complex parsing (IP addresses with comma separators, MAC bytes)
  • Confirmed integration points in Arduino code (function calls, variable usage)
  • Added validation logic for critical configuration parameters

Documentation ✅

  • Server: Updated INSTALLATION.md and SD_CARD_CONFIG.md to emphasize required nature
  • Client: Updated INSTALLATION.md to reflect SD card configuration as mandatory
  • Updated example configurations to reflect mandatory nature and limits
  • Added warnings about device halt behavior when config missing
  • Clear instructions for field configuration management for both server and client

Usage Example

Before (graceful fallback):

  • Server/Client would start with compile-time defaults if SD card missing
  • Could operate with placeholder configuration values

After (strict requirement with validation):

# server_config.txt on SD card - REQUIRED for server
HOLOGRAM_DEVICE_KEY=actual_device_key_here
DAILY_EMAIL_HOUR=8
SERVER_LOCATION=Tank Farm B
MAX_REPORTS_IN_MEMORY=50  # Maximum: 100 (will be capped if exceeded)

# tank_config.txt on SD card - REQUIRED for client  
HOLOGRAM_DEVICE_KEY=actual_device_key_here
SITE_NAME=Tank Farm B
ALARM_PHONE_PRIMARY=+15551234567

Both server and client will HALT with error messages if:

  • SD card not inserted
  • Configuration file missing
  • Critical settings not configured

YOLO approach with safety: No safety net for missing config, but automatic bounds checking prevents buffer overflow!

Fixes #39.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Copilot AI changed the title [WIP] config settings moved to sd card Move server configuration settings to SD card for field updates Sep 22, 2025
Copilot AI requested a review from dorkmo September 22, 2025 16:39
@dorkmo
Copy link
Collaborator

dorkmo commented Sep 22, 2025

@copilot let's YOLO it and remove
"Graceful fallback to compile-time defaults if SD card config unavailable"
And assume SD card config will be available

Copy link
Contributor Author

Copilot AI commented Sep 22, 2025

@copilot let's YOLO it and remove
"Graceful fallback to compile-time defaults if SD card config unavailable"
And assume SD card config will be available

Done! Removed the graceful fallback behavior. The server now requires SD card configuration and will halt with error messages if:

  • SD card is not inserted or cannot be initialized
  • server_config.txt file is missing from SD card
  • HOLOGRAM_DEVICE_KEY is not configured or still has default placeholder value

Changes in commit 79187e2:

  • Removed all fallback #define values from server_config.h
  • Updated config variables to use basic defaults instead of compile-time constants
  • Modified loadServerConfigurationFromSD() to halt execution if SD card config unavailable
  • Added validation to ensure critical settings like device key are properly configured
  • Updated documentation to reflect that SD card config is now mandatory

The YOLO approach is now in effect - no safety net, SD card configuration must be present and valid! 🚀

Copilot AI changed the title Move server configuration settings to SD card for field updates Require SD card configuration for server settings (remove fallback defaults) Sep 22, 2025
@dorkmo
Copy link
Collaborator

dorkmo commented Sep 22, 2025

@copilot can you now make the client operate the same way as the changes made to the server. We can come back and change how the sensors pins are reserved later.

Copy link
Contributor Author

Copilot AI commented Sep 22, 2025

@copilot can you now make the client operate the same way as the changes made to the server. We can come back and change how the sensors pins are reserved later.

Done! Applied the same SD card requirement changes to the client. The client now operates exactly like the server changes:

Client Changes in commit b0d3498:

  • Removed all fallback #define values from config_template.h
  • Updated 20+ configuration variables to use basic defaults instead of compile-time constants
  • Modified loadSDCardConfiguration() to halt execution if SD card config unavailable
  • Added validation for critical settings like HOLOGRAM_DEVICE_KEY and ALARM_PHONE_PRIMARY
  • Updated tank_config_example.txt to emphasize required nature
  • Updated client installation guide to reflect mandatory SD card configuration

Client will now HALT with error messages if:

  • SD card is not inserted or cannot be initialized
  • tank_config.txt file is missing from SD card
  • HOLOGRAM_DEVICE_KEY is not configured
  • ALARM_PHONE_PRIMARY is not configured

Sensor pins kept as compile-time constants as requested - all the hardware-specific pin definitions, sensor types, and I2C addresses remain in config.h since they're hardware-dependent and can be changed later.

Both server and client now have the same "YOLO" approach - no safety net, SD card configuration must be present and valid! 🚀

Copilot AI changed the title Require SD card configuration for server settings (remove fallback defaults) Require SD card configuration for both server and client settings (remove fallback defaults) Sep 22, 2025
@dorkmo dorkmo marked this pull request as ready for review September 22, 2025 20:06
Copilot AI review requested due to automatic review settings September 22, 2025 20:06
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements mandatory SD card-based configuration for both Tank Alarm Server and Client, removing all fallback defaults to ensure consistent field-configurable operation. The change enables configuration updates without code recompilation by requiring SD card files at startup.

  • Migrated all configuration from compile-time constants to runtime SD card files
  • Added mandatory configuration validation with device halt on missing/invalid settings
  • Updated documentation to reflect the new "SD card required" approach

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
server_config.txt New SD card configuration template for server with all required settings
server_config.h Stripped down to hardware constants only, removed fallback configuration values
TankAlarm-092025-Server-Hologram.ino Added loadServerConfigurationFromSD() function and mandatory validation logic
SD_CARD_CONFIG.md New documentation for required SD card configuration process
INSTALLATION.md (Server) Updated to emphasize mandatory SD card configuration
tank_config_example.txt Updated client configuration template with "REQUIRED" emphasis
config_template.h Stripped down to hardware constants only for client
TankAlarm-092025-Client-Hologram.ino Updated loadSDCardConfiguration() to require SD card with validation
INSTALLATION.md (Client) Updated to reflect mandatory SD card configuration approach

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

unsigned long lastDailyEmailRetry = 0;
const unsigned long DAILY_EMAIL_RETRY_INTERVAL = 1800000; // 30 minutes in milliseconds
const int MAX_TANK_REPORTS = 50; // Maximum number of reports to store in memory
const int MAX_TANK_REPORTS = 100; // Maximum array size (actual limit from configuration)
Copy link

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment suggests this is the maximum array size but the actual limit comes from configuration. This could be confusing since the array is statically allocated at 100 but maxReportsInMemory can be configured to a different value. Consider renaming to clarify this is the hard limit: // Hard array limit (configured limit must not exceed this)

Suggested change
const int MAX_TANK_REPORTS = 100; // Maximum array size (actual limit from configuration)
const int MAX_TANK_REPORTS = 100; // Hard array limit (configured limit must not exceed this)

Copilot uses AI. Check for mistakes.

// Store report in memory
if (reportCount < MAX_TANK_REPORTS) {
if (reportCount < maxReportsInMemory && reportCount < MAX_TANK_REPORTS) {
Copy link

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition checks both maxReportsInMemory and MAX_TANK_REPORTS but doesn't validate that maxReportsInMemory <= MAX_TANK_REPORTS during configuration loading. If maxReportsInMemory is set higher than MAX_TANK_REPORTS in the config file, this will silently cap at the lower value without warning the user.

Copilot uses AI. Check for mistakes.
Comment on lines 1467 to 1494
int pos = 0;
for (int i = 0; i < 4; i++) {
int nextPos = value.indexOf(',', pos);
if (nextPos == -1 && i < 3) break;
String octet = (nextPos == -1) ? value.substring(pos) : value.substring(pos, nextPos);
staticIpAddress[i] = octet.toInt();
pos = nextPos + 1;
}
} else if (key == "STATIC_GATEWAY") {
// Parse comma-separated gateway address
int pos = 0;
for (int i = 0; i < 4; i++) {
int nextPos = value.indexOf(',', pos);
if (nextPos == -1 && i < 3) break;
String octet = (nextPos == -1) ? value.substring(pos) : value.substring(pos, nextPos);
staticGateway[i] = octet.toInt();
pos = nextPos + 1;
}
} else if (key == "STATIC_SUBNET") {
// Parse comma-separated subnet mask
int pos = 0;
for (int i = 0; i < 4; i++) {
int nextPos = value.indexOf(',', pos);
if (nextPos == -1 && i < 3) break;
String octet = (nextPos == -1) ? value.substring(pos) : value.substring(pos, nextPos);
staticSubnet[i] = octet.toInt();
pos = nextPos + 1;
}
Copy link

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This IP address parsing logic is duplicated three times (for static IP, gateway, and subnet). Consider extracting this into a helper function like parseCommaSeparatedIP(String value, int* targetArray) to reduce code duplication and make it easier to maintain.

Suggested change
int pos = 0;
for (int i = 0; i < 4; i++) {
int nextPos = value.indexOf(',', pos);
if (nextPos == -1 && i < 3) break;
String octet = (nextPos == -1) ? value.substring(pos) : value.substring(pos, nextPos);
staticIpAddress[i] = octet.toInt();
pos = nextPos + 1;
}
} else if (key == "STATIC_GATEWAY") {
// Parse comma-separated gateway address
int pos = 0;
for (int i = 0; i < 4; i++) {
int nextPos = value.indexOf(',', pos);
if (nextPos == -1 && i < 3) break;
String octet = (nextPos == -1) ? value.substring(pos) : value.substring(pos, nextPos);
staticGateway[i] = octet.toInt();
pos = nextPos + 1;
}
} else if (key == "STATIC_SUBNET") {
// Parse comma-separated subnet mask
int pos = 0;
for (int i = 0; i < 4; i++) {
int nextPos = value.indexOf(',', pos);
if (nextPos == -1 && i < 3) break;
String octet = (nextPos == -1) ? value.substring(pos) : value.substring(pos, nextPos);
staticSubnet[i] = octet.toInt();
pos = nextPos + 1;
}
parseCommaSeparatedIP(value, staticIpAddress);
} else if (key == "STATIC_GATEWAY") {
// Parse comma-separated gateway address
parseCommaSeparatedIP(value, staticGateway);
} else if (key == "STATIC_SUBNET") {
// Parse comma-separated subnet mask
parseCommaSeparatedIP(value, staticSubnet);

Copilot uses AI. Check for mistakes.
}
}

if (alarmPhonePrimary.length() == 0 || alarmPhonePrimary == "+12223334444") {
Copy link

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded placeholder value +12223334444 should be defined as a constant rather than a magic string. If this placeholder changes in the config template, this validation could miss it. Consider defining #define DEFAULT_PHONE_PLACEHOLDER \"+12223334444\" or validate against the actual template file content.

Copilot uses AI. Check for mistakes.
@dorkmo
Copy link
Collaborator

dorkmo commented Oct 8, 2025

@copilot This condition checks both maxReportsInMemory and MAX_TANK_REPORTS but doesn't validate that maxReportsInMemory <= MAX_TANK_REPORTS during configuration loading. If maxReportsInMemory is set higher than MAX_TANK_REPORTS in the config file, this will silently cap at the lower value without warning the user.

Copy link
Contributor Author

Copilot AI commented Oct 8, 2025

@copilot This condition checks both maxReportsInMemory and MAX_TANK_REPORTS but doesn't validate that maxReportsInMemory <= MAX_TANK_REPORTS during configuration loading. If maxReportsInMemory is set ...

Fixed! Added validation to prevent buffer overflow when maxReportsInMemory exceeds the array size.

Changes in commit 5f4a0e0:

  • Added bounds checking in loadServerConfigurationFromSD()
  • If MAX_REPORTS_IN_MEMORY in config exceeds MAX_TANK_REPORTS (100), it's automatically capped
  • Warning message displayed when value is capped
  • Updated server_config.txt and SD_CARD_CONFIG.md to document the 100 limit

This prevents potential buffer overflow if someone configures a value larger than the allocated array size.

@dorkmo dorkmo requested a review from Copilot October 8, 2025 20:24
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +1464 to +1469
if (maxReportsInMemory > MAX_TANK_REPORTS) {
Serial.println("WARNING: MAX_REPORTS_IN_MEMORY (" + String(maxReportsInMemory) +
") exceeds MAX_TANK_REPORTS (" + String(MAX_TANK_REPORTS) + ")");
Serial.println("Capping maxReportsInMemory to " + String(MAX_TANK_REPORTS));
maxReportsInMemory = MAX_TANK_REPORTS;
}
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The bounds checking logic could be extracted into a separate validation function to improve code organization and reusability, especially since similar validation patterns may be needed for other configuration parameters.

Copilot uses AI. Check for mistakes.
Comment on lines +1472 to +1501
} else if (key == "STATIC_IP_ADDRESS") {
// Parse comma-separated IP address like "192,168,1,100"
int octetCount = 0;
int lastPos = 0;
int commaPos = -1;
byte tempIp[4];
bool validIp = true;
for (int i = 0; i < 4; i++) {
commaPos = value.indexOf(',', lastPos);
String octetStr;
if (i < 3) {
if (commaPos == -1) {
validIp = false;
break;
}
octetStr = value.substring(lastPos, commaPos);
lastPos = commaPos + 1;
} else {
octetStr = value.substring(lastPos);
}
int octet = octetStr.toInt();
if (octet < 0 || octet > 255) {
validIp = false;
break;
}
tempIp[i] = (byte)octet;
}
if (validIp) {
for (int i = 0; i < 4; i++) staticIpAddress[i] = tempIp[i];
}
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IP address parsing logic is duplicated for STATIC_IP_ADDRESS, STATIC_GATEWAY, and STATIC_SUBNET. This code duplication should be extracted into a helper function that returns a parsed IP array to improve maintainability and reduce the risk of inconsistent behavior.

Copilot uses AI. Check for mistakes.
daysToKeepLogs = value.toInt();
} else if (key == "STATIC_IP_ADDRESS") {
// Parse comma-separated IP address like "192,168,1,100"
int octetCount = 0;
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable octetCount is declared but never used in the IP address parsing logic. This unused variable should be removed to clean up the code.

Suggested change
int octetCount = 0;

Copilot uses AI. Check for mistakes.
@dorkmo
Copy link
Collaborator

dorkmo commented Oct 8, 2025

@copilot
The bounds checking logic could be extracted into a separate validation function to improve code organization and reusability, especially since similar validation patterns may be needed for other configuration parameters.

The IP address parsing logic is duplicated for STATIC_IP_ADDRESS, STATIC_GATEWAY, and STATIC_SUBNET. This code duplication should be extracted into a helper function that returns a parsed IP array to improve maintainability and reduce the risk of inconsistent behavior.

The variable octetCount is declared but never used in the IP address parsing logic. This unused variable should be removed to clean up the code.

@dorkmo
Copy link
Collaborator

dorkmo commented Oct 8, 2025

@copilot The bounds checking logic could be extracted into a separate validation function to improve code organization and reusability, especially since similar validation patterns may be needed for other configuration parameters.

The IP address parsing logic is duplicated for STATIC_IP_ADDRESS, STATIC_GATEWAY, and STATIC_SUBNET. This code duplication should be extracted into a helper function that returns a parsed IP array to improve maintainability and reduce the risk of inconsistent behavior.

The variable octetCount is declared but never used in the IP address parsing logic. This unused variable should be removed to clean up the code.

@dorkmo
Copy link
Collaborator

dorkmo commented Oct 8, 2025

@copilot The IP address parsing logic is duplicated for STATIC_IP_ADDRESS, STATIC_GATEWAY, and STATIC_SUBNET. This code duplication should be extracted into a helper function that returns a parsed IP array to improve maintainability and reduce the risk of inconsistent behavior.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines 587 to 589
if (MONTHLY_REPORT_ENABLED) {
client.println("<p>Monthly Reports: <span style='color: green;'>Enabled</span> (Day " + String(MONTHLY_REPORT_DAY) + " at " + String(MONTHLY_REPORT_HOUR) + ":00)</p>");
client.println("<p>Monthly Reports: <span style='color: green;'>Enabled</span> (Day " + String(monthlyReportDay) + " at " + String(monthlyReportHour) + ":00)</p>");
}
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code references MONTHLY_REPORT_ENABLED in the #ifdef directive but uses monthlyReportEnabled variable in the condition check. This creates inconsistent behavior where the compile-time check may not match the runtime configuration.

Copilot uses AI. Check for mistakes.
Comment on lines +1477 to +1498
byte tempIp[4];
bool validIp = true;
for (int i = 0; i < 4; i++) {
commaPos = value.indexOf(',', lastPos);
String octetStr;
if (i < 3) {
if (commaPos == -1) {
validIp = false;
break;
}
octetStr = value.substring(lastPos, commaPos);
lastPos = commaPos + 1;
} else {
octetStr = value.substring(lastPos);
}
int octet = octetStr.toInt();
if (octet < 0 || octet > 255) {
validIp = false;
break;
}
tempIp[i] = (byte)octet;
}
Copy link

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IP address parsing logic is duplicated three times for static IP, gateway, and subnet. This should be extracted into a reusable parseIPAddress() helper function to reduce code duplication.

Copilot uses AI. Check for mistakes.
@dorkmo
Copy link
Collaborator

dorkmo commented Oct 8, 2025

@copilot The code references MONTHLY_REPORT_ENABLED in the #ifdef directive but uses monthlyReportEnabled variable in the condition check. This creates inconsistent behavior where the compile-time check may not match the runtime configuration.

@dorkmo
Copy link
Collaborator

dorkmo commented Oct 9, 2025

@copilot The IP address parsing logic is duplicated three times for static IP, gateway, and subnet. This should be extracted into a reusable parseIPAddress() helper function to reduce code duplication.

@dorkmo
Copy link
Collaborator

dorkmo commented Oct 9, 2025

@copilot The code references MONTHLY_REPORT_ENABLED in the #ifdef directive but uses monthlyReportEnabled variable in the condition check. This creates inconsistent behavior where the compile-time check may not match the runtime configuration.

@dorkmo dorkmo merged commit 8f543c7 into master Oct 9, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

config settings moved to sd card

2 participants