Merge pull request #282 from akshaylg0314/metric

feat(api)-Update Metrics strucutre,Load ContainerInfo from ETCD and Define more REST APIs on SettingsService

Author: youngtaekiim

src/server/settingsservice/README.md

@@ -1,15 +1,15 @@
 # PICCOLO Settings Service
 
-The Settings Service is a core component of the PICCOLO framework that provides centralized configuration management and metrics filtering capabilities.
+The Settings Service is a core component of the PICCOLO framework that provides centralized configuration management and metrics filtering capabilities for vehicle service orchestration.
 
 ## Features
 
 - **Configuration Management**: Create, read, update, and delete YAML/JSON configurations
 - **Schema Validation**: Validate configurations against JSON schemas
 - **Change History**: Track configuration changes with rollback capabilities
-- **Metrics Filtering**: Filter and serve monitoring metrics from ETCD
+- **Metrics Retrieval**: Retrieve and filter monitoring metrics from ETCD (NodeInfo, ContainerInfo, SocInfo, BoardInfo)
 - **Multiple Interfaces**: REST API and CLI interfaces
-- **ETCD Integration**: Uses ETCD as the backend storage
+- **ETCD Integration**: Direct integration with monitoring ETCD storage for real-time vehicle orchestration data
 
 ## Architecture
 
@@ -18,22 +18,22 @@ The Settings Service consists of the following modules:
 - `settings_core`: Service initialization and coordination
 - `settings_config`: Configuration management with YAML/JSON support
 - `settings_history`: Change history tracking and rollback
-- `settings_monitoring`: Metrics data filtering from ETCD
-- `settings_storage`: ETCD client for data persistence
-- `settings_api`: REST API server
-- `settings_cli`: Command-line interface
-- `settings_utils`: Common utilities
+- `settings_monitoring`: High-level metrics data retrieval and filtering with caching
+- `monitoring_etcd`: Direct ETCD operations for monitoring data (`/piccolo/metrics/`, `/piccolo/logs/`)
+- `monitoring_types`: Type definitions for vehicle orchestration metrics (NodeInfo, SocInfo, BoardInfo)
+- `settings_storage`: ETCD client for configuration data persistence
+- `settings_api`: REST API server with comprehensive metrics endpoints
+- `settings_cli`: Command-line interface with interactive shell
+- `settings_utils`: Common utilities (error handling, logging, YAML processing)
 
 ## Building
 
-```bash
 # Build the settings service
 cd src/server/settingsservice
 cargo build
 
 # Or build the entire project
 make build
-```
 
 ## Running
 
@@ -74,13 +74,25 @@ The Settings Service provides a comprehensive REST API:
 - `DELETE /api/v1/settings/{path}` - Delete configuration
 - `POST /api/v1/settings/validate` - Validate configuration
 
-### Metrics Management
+### Metrics Management (Vehicle Orchestration)
 
-- `GET /api/v1/metrics` - Get filtered metrics
-- `GET /api/v1/metrics/{id}` - Get specific metric
-- `GET /api/v1/metrics/component/{component}` - Get metrics by component
+**Enhanced endpoints with direct ETCD access:**
+
+- `GET /api/v1/metrics` - Get all metrics from ETCD with optional filtering
+- `GET /api/v1/metrics/nodes` - Get all node metrics (NodeInfo) - **FIXED**
+- `GET /api/v1/metrics/containers` - Get all container metrics (ContainerInfo) - **FIXED**
+- `GET /api/v1/metrics/socs` - Get all SoC metrics (SocInfo) - **FIXED**
+- `GET /api/v1/metrics/boards` - Get all board metrics (BoardInfo) - **FIXED**
+- `GET /api/v1/metrics/nodes/{node_name}` - Get specific node metric - **FIXED**
+- `GET /api/v1/metrics/containers/{container_id}` - Get specific container metric - **FIXED**
 - `GET /api/v1/metrics/filters` - List metric filters
 - `POST /api/v1/metrics/filters` - Create metric filter
+- `DELETE /api/v1/metrics/{component}/{id}` - Delete specific metric
+
+**Query parameters for filtering:**
+- `?component=node|container|soc|board` - Filter by component type
+- `?max_items=N` - Limit number of results
+- `?metric_type=NodeInfo|ContainerInfo|SocInfo|BoardInfo` - Filter by metric type
 
 ### History Management
 
@@ -107,13 +119,16 @@ config delete <path>           # Delete configuration
 config validate <path>         # Validate configuration
 ```
 
-### Metrics Commands
+### Metrics Commands (Vehicle Orchestration)
 
 ```bash
-metrics list                   # List all metrics
-metrics get <id>               # Get specific metric
-metrics filter <component>     # Filter metrics by component
-metrics filters                # List all filters
+metrics nodes                  # List all node metrics
+metrics containers             # List all container metrics  
+metrics socs                   # List all SoC metrics
+metrics boards                 # List all board metrics
+metrics node <name>            # Get specific node metric
+metrics container <id>         # Get specific container metric
+metrics stats                  # Show metrics statistics
 ```
 
 ### History Commands
@@ -132,6 +147,7 @@ The service can be configured using command-line arguments or environment variab
 - `--bind-address`: HTTP server bind address (default: `0.0.0.0`)
 - `--bind-port`: HTTP server bind port (default: `8080`)
 - `--log-level`: Log level (default: `info`)
+- `--cli`: Run in CLI mode instead of server mode
 
 ## Testing
 
@@ -148,33 +164,97 @@ cargo test -- --nocapture
 ### Create a Configuration
 
 ```bash
-curl -X POST http://localhost:8080/api/v1/settings/myapp/config \
+curl -X POST http://localhost:8080/api/v1/settings/vehicle/orchestrator \
   -H "Content-Type: application/json" \
   -d '{
     "content": {
-      "database": {
-        "host": "localhost",
-        "port": 5432
+      "node_selection": {
+        "strategy": "resource_based",
+        "cpu_threshold": 80.0,
+        "memory_threshold": 90.0
+      },
+      "container_policy": {
+        "restart_policy": "always",
+        "resource_limits": {
+          "cpu": "2.0",
+          "memory": "4Gi"
+        }
       }
     },
-    "schema_type": "database-config",
-    "author": "admin",
-    "comment": "Initial database configuration"
+    "schema_type": "orchestrator-config",
+    "author": "vehicle-admin",
+    "comment": "Vehicle service orchestration configuration"
   }'
 ```
 
 ### Get Configuration
 
 ```bash
-curl http://localhost:8080/api/v1/settings/myapp/config
+curl http://localhost:8080/api/v1/settings/vehicle/orchestrator
+```
+
+### Get All Node Metrics - **WORKING**
+
+```bash
+curl http://localhost:8080/api/v1/metrics/nodes
+```
+
+### Get Specific Node Metrics
+
+```bash
+curl http://localhost:8080/api/v1/metrics/nodes/vehicle-ecu-01
+```
+
+### Get All Container Metrics - **WORKING**
+
+```bash
+curl http://localhost:8080/api/v1/metrics/containers
 ```
 
-### Filter Metrics
+### Get Container Metrics for Specific Container
 
 ```bash
-curl "http://localhost:8080/api/v1/metrics?component=nodeagent&metric_type=gauge"
+curl http://localhost:8080/api/v1/metrics/containers/db368045fa4d40ffa3ba8cae61eeb9df36e120a2350e36d71e547b7ce3f1a9d5
 ```
 
+### Filter Metrics with Query Parameters
+
+```bash
+# Get only node metrics, limit to 10 items
+curl "http://localhost:8080/api/v1/metrics?component=node&max_items=10"
+
+# Get all container metrics
+curl "http://localhost:8080/api/v1/metrics?component=container"
+
+# Get all SoC metrics
+curl "http://localhost:8080/api/v1/metrics/socs"
+
+# Get all board metrics
+curl "http://localhost:8080/api/v1/metrics/boards"
+```
+
+### Enable Debug Logging for Troubleshooting
+
+```bash
+RUST_LOG=debug ./target/debug/settingsservice
+```
+
+## Vehicle Service Orchestration Integration
+
+The Settings Service integrates directly with the Pullpiri vehicle orchestration framework:
+
+- **MonitoringServer**: Stores vehicle node, container, SoC, and board metrics in ETCD at `/piccolo/metrics/`
+- **NodeAgent**: Reports node resource utilization and container status to MonitoringServer
+- **APIServer**: Consumes configurations for orchestration policies
+- **ETCD**: Central storage for both configurations (`/piccolo/settings/`) and real-time metrics (`/piccolo/metrics/`)
+
+## Port Usage
+
+Following Pullpiri networking conventions:
+- **Settings Service**: `8080` (configurable within Pullpiri's 47001-47099 range)
+- **ETCD**: `2379, 2380` (standard ETCD ports)
+- **Other Pullpiri Services**: `47001-47099` (gRPC: 47001+, REST: up to 47099)
+
 ## Dependencies
 
 - Rust 1.70+
@@ -183,4 +263,4 @@ curl "http://localhost:8080/api/v1/metrics?component=nodeagent&metric_type=gauge
 
 ## License
 
-Apache-2.0
+Apache-2.0 (following Pullpiri framework licensing)

src/server/settingsservice/src/monitoring_etcd.rs

@@ -4,6 +4,7 @@
 //! Integration with monitoring server's etcd storage
 
 use crate::monitoring_types::{BoardInfo, NodeInfo, SocInfo};
+use common::monitoringserver::ContainerInfo;
 use serde::{de::DeserializeOwned, Serialize};
 use thiserror::Error;
 use tracing::{debug, warn};
@@ -149,6 +150,26 @@ pub async fn delete_board_info(board_id: &str) -> Result<()> {
     delete_info("boards", board_id).await
 }
 
+/// Store ContainerInfo in etcd
+pub async fn store_container_info(container_info: &ContainerInfo) -> Result<()> {
+    store_info("containers", &container_info.id, container_info).await
+}
+
+/// Get ContainerInfo from etcd
+pub async fn get_container_info(container_id: &str) -> Result<ContainerInfo> {
+    get_info("containers", container_id).await
+}
+
+/// Get all containers from etcd
+pub async fn get_all_containers() -> Result<Vec<ContainerInfo>> {
+    get_all_info("containers").await
+}
+
+/// Delete ContainerInfo from etcd
+pub async fn delete_container_info(container_id: &str) -> Result<()> {
+    delete_info("containers", container_id).await
+}
+
 /// Get node logs from etcd
 pub async fn get_node_logs(node_name: &str) -> Result<Vec<String>> {
     get_logs("nodes", node_name).await
@@ -164,6 +185,11 @@ pub async fn get_board_logs(board_id: &str) -> Result<Vec<String>> {
     get_logs("boards", board_id).await
 }
 
+/// Get container logs from etcd
+pub async fn get_container_logs(container_id: &str) -> Result<Vec<String>> {
+    get_logs("containers", container_id).await
+}
+
 /// Generic function to get logs from etcd
 async fn get_logs(resource_type: &str, resource_id: &str) -> Result<Vec<String>> {
     let prefix = format!("/piccolo/logs/{}/{}", resource_type, resource_id);
@@ -200,6 +226,14 @@ pub async fn store_board_metadata(board_id: &str, metadata: &serde_json::Value)
     store_metadata("boards", board_id, metadata).await
 }
 
+/// Store container metadata in etcd
+pub async fn store_container_metadata(
+    container_id: &str,
+    metadata: &serde_json::Value,
+) -> Result<()> {
+    store_metadata("containers", container_id, metadata).await
+}
+
 /// Generic function to store metadata in etcd
 async fn store_metadata(
     resource_type: &str,