Merge pull request #397 from VirtualMetric:DT-557-navattic-directors

DT-557-Add demo components for directors configuration and management

Author: KorayErkan

demos.json

@@ -4,7 +4,9 @@
   "users-delete": "cmk287alg000304jrgque5e5u",
   "directors-create": "cmk3zld8y000004l5c6250jjt",
   "directors-manage": "cmkqti82k000004lachjrcrxd",
+  "directors-update": "cmlt6uaqo000004l1b7mtebs0",
   "clusters-create": "cmk5d6u1y000004k06x5q5c4g",
+  "clusters-manage": "cmlt74lou000604joclwi40p7",
   "devices-create": "cmkdpnivn000004jr7jtghtww",
   "onboarding": "cml958r58000204jp25ip65q7",
   "settings-company": "cmlkotkwb002304iffzn99t07",

docs/configuration/directors/clusters.mdx

@@ -2,10 +2,16 @@
 sidebar_label: Clusters
 ---
 
-# Director Clusters
+# Directors: Clusters
 
 Clusters group multiple Directors together for high availability. When one Director fails, the remaining Directors automatically take over its workload. <Topic id="devices-agents">Agents</Topic> and Devices connect to the cluster as a whole, ensuring continuous operation as long as at least one Director remains healthy.
 
+> Provider → Device → Preprocessing → Pipeline → Postprocessing → Target → Consumer
+>
+> **Cluster** _(groups Directors for high availability and automatic failover)_
+
+<Image id="clusters-flow" />
+
 :::warning[Permanent Assignment]
 Once a Director is added to a cluster, it cannot be individually modified, updated, or removed. All configuration changes must be made at the cluster level. The only way to release a Director from a cluster is to delete the entire cluster, after which the Director returns to standalone status.
 :::
@@ -34,6 +40,8 @@ Filter clusters using **Search clusters** (by name) and **Status** dropdown (All
 
 The cluster creation process guides you through a 3-step wizard.
 
+<Demo id="clusters-create" aspectRatio="18/9" />
+
 #### Step 1: Cluster Details
 
 **Basic Configuration:**

docs/configuration/directors/deployment.mdx

@@ -6,15 +6,7 @@ sidebar_label: Deployment
 
 **VirtualMetric DataStream Directors** support flexible deployment options to match your infrastructure requirements and operational preferences. Whether you're running on physical hardware, virtual machines, or containerized environments, **Directors** can be deployed to optimize performance while maintaining data sovereignty.
 
-## Definitions
-
-**Directors** are lightweight, containerized services that process security telemetry data locally while connecting to the **DataStream** cloud platform for configuration management. This architecture ensures your sensitive data remains within your controlled environment while providing centralized management capabilities.
-
-### Supported Models
-
-A **Standalone Director** uses a single instance to handle all data processing. This model is recommended for most production deployments due to its simple configuration and management. It is suitable for small to medium-scale environments.
-
-A **Clustered Director** deployment uses multiple instances with load balancing and high availability. Clusters provide automatic failover and redundancy, horizontal scaling capabilities, and are ideal for mission-critical, high-volume environments. To create a cluster, first deploy Directors as standalone instances, then group them via the **Clusters** tab in the Directors management interface. See <Topic id="directors-management">Directors: Management</Topic> for cluster configuration details.
+For Director concepts, capabilities, and installation types, see <Topic id="directors-overview">Directors: Overview</Topic>.
 
 ## Options
 
@@ -94,81 +86,101 @@ Windows deployments provide native Windows service integration with Active Direc
 - Organizations requiring agentless Windows monitoring
 - Environments with existing Windows management infrastructure
 
-## Agent Pre-Processing Architecture
+## Installation Process
+
+### Standalone Director Installation
+
+<Demo id="directors-create" aspectRatio="16/9" />
 
-**VirtualMetric Agents** support optional pipeline-based pre-processing before sending data to **Directors**. This distributed processing model reduces **Director** workload and enables edge-based data transformation.
+:::info
+Directors are always created as standalone instances. To create a clustered deployment for high availability, first create Directors using this wizard, then group them using the **Clusters** tab. See <Topic id="directors-clusters">Clusters</Topic> for configuration details.
+:::
 
-### Processing Models
+Navigate to **Home** > **Fleet Management** > **Directors** and click <gui>Create director</gui> to begin the 3-step setup wizard.
 
-In the **Traditional Model**, the **Agent** collects logs locally at the endpoint and sends raw data to the **Director**. The **Director** then processes data through pipelines and forwards the processed data to targets.
+#### Step 1: Director Setup
 
-In the **Pre-Processing Model**, the **Agent** collects logs locally at the endpoint and processes data through configured pipelines before sending pre-processed data to the **Director**. The **Director** forwards data to targets, with optional additional processing if needed.
+**Director Name:**
 
-### Pre-Processing Benefits
+Enter a unique identifier for the Director (3-64 characters). This name is used for identification across the platform.
 
-Pre-processing reduces **Director** processing load through distributed computation and lowers network bandwidth consumption via edge-based filtering and transformation. This approach improves scalability for large-scale deployments with multiple **Agents** and enables faster data delivery through parallel processing at collection points.
+**Mode Selection:**
 
-From an architectural perspective, edge-based filtering reduces unnecessary data transmission while local transformation enables compliance requirements at the data source. The distributed processing model supports horizontal scaling and reduces central processing bottlenecks in high-volume environments.
+Select the management approach using radio tiles:
 
-### Pre-Processing Configuration
+- **Managed** - VirtualMetric manages Director infrastructure with automated updates, proactive monitoring, and reduced operational overhead
+- **Self-Managed** - User controls Director deployment with manual updates, custom security configurations, and full infrastructure control
 
-**Agent** pre-processing is configured through the **Director's** device configuration for that **Agent**. Pipelines assigned to **Agent** devices execute locally on the **Agent**, using the same pipeline syntax and processors available as **Director** pipelines. Configuration is managed centrally through the **Director** for consistency.
+:::info[Self-Managed Directors]
+A self-managed Director is indicated under the **Mode** column as _Self-managed_, with a warning icon when configuration has changed. Hovering over the icon displays a tooltip reading "Configuration has changed and the current one has to be deployed."
 
-:::tip
-**Agent** pipelines support hot configuration reload. Changes made in the **Director** interface are synchronized to **Agents** automatically without requiring an **Agent** restart.
+The actions menu contains a <gui>Download Config</gui> option that downloads the `vmetric.vmf` file. Place this file under the `<vm_root>\Director\config` directory. Self-managed Directors skip access verification — monitor errors through the CLI or the files under `<vm_root>\Director\storage\logs`.
 :::
 
-### Use Cases for Agent Pre-Processing
+#### Step 2: Access Management
 
-In **high-volume environments**, you can filter non-essential logs at the collection point before transmission, reduce network bandwidth for high-volume log sources, and distribute processing load across multiple **Agent** endpoints.
+Configure optional access controls for the Director.
 
-For **compliance and privacy**, mask sensitive data (PII, credentials) at the source before transmission. Apply regulatory transformations at the data collection point to ensure data compliance before leaving the endpoint network.
+**Access Tokens:**
 
-In **edge computing** scenarios, process data locally in remote or branch offices to minimize data transmission to the central **Director**. This approach supports disconnected or intermittent connectivity scenarios.
+- **Enable Access Tokens** - Toggle to require token authentication
+- **Token Name** - Identifier for the token
+- **Token Value** - Auto-generated when token is created (use copy button to copy)
+- **Expiration** - Token validity period (default: 3 months):
+  - 1 month
+  - 3 months
+  - 1 year
+  - Unlimited
 
-For **cost optimization**, reduce **Director** infrastructure requirements through distributed processing. Lower network bandwidth costs via edge-based filtering and optimize central processing capacity allocation.
+**Access Restriction:**
 
-### Configuration Considerations
+Restrict which IP addresses can connect to the Director:
 
-When implementing **Agent** pre-processing, balance processing load between **Agents** and **Directors** based on infrastructure capacity. Consider network latency and bandwidth when deciding what to process at the edge. Use **Agent** pre-processing for filtering and basic transformations, reserving complex processing (enrichment, external lookups) for the **Director** when possible. Monitor **Agent** resource utilization to prevent endpoint performance impact.
+- **Enable Access Restriction** - Toggle to enable IP filtering
+- **Restriction Type**:
+  - **Single Block** - Single IP address
+  - **Range** - IP address range (start-end)
+  - **CIDR** - CIDR notation (e.g., 10.0.0.0/8)
+- **Sources** - List of allowed IP addresses/ranges
 
-## Installation Process
+#### Step 3: Connect Director
 
-### Standalone Director Installation
+**Installation Scripts:**
+- **PowerShell Script** - For Windows deployments
+- **Bash Script** - For Linux/macOS deployments
+- **Copy Button** - One-click copy to clipboard
+- **API Key** - Auto-generated secure authentication key included in scripts
 
-The standard installation process follows a guided setup through the **DataStream** web interface:
+Run the provided script with administrative privileges on the target system. The installation downloads and configures the Director service with automatic service registration and startup configuration.
 
-1. **Access Director Creation**
-   - Navigate to Home > Fleet Management > Directors
-   - Click "Create director" to begin setup process
+**Connection Verification:**
 
-2. **Configure Director Properties**
-   - Assign unique Director name for identification
-   - Select "Standalone" installation type
-   - Choose appropriate platform
+After running the installation script, verify Director connectivity:
 
-      A self-managed director is indicated under the **Mode** column as _Self-managed_, with a warning icon to its right. Hovering over the icon displays a tooltip, informing the user that the xonfiguration has changed and that the current one has to be deployed.
+- **Connecting** - Director attempting initial connection
+- **Success** - Director successfully connected to platform
+- **Failed** - Connection attempt unsuccessful
 
-      :::info
-      The actions menu of a self-managed director contains a _Download config_ option. Clicking it downloads the `vmetric.vmf` file to the **Downloads** directory of **Windows**. This file should be placed under the `<vm_root>\Director\config` directory.
-      :::
+Real-time connection status updates with visual indicators for each state.
 
-      This option removes the access verification step. The user can monitor errors through the CLI or the files under the `<vm_root>\Director\storage\logs` directory.
+**Completion Options:**
 
-3. **Generate Installation Scripts**
-   - System generates platform-specific installation scripts
-   - Unique API key created for secure cloud connectivity
-   - Scripts provided for both PowerShell (Windows) and Bash (Linux)
+**Connected Successfully:**
+- <gui>Complete Setup</gui> - Finalize and navigate to Director detail view
 
-4. **Execute Installation**
-   - Run provided script with administrative privileges on target system
-   - Installation downloads and configures Director service
-   - Automatic service registration and startup configuration
+**Not Yet Connected:**
+- <gui>Connect Later</gui> - Save Director configuration without connection verification
+- Return later to verify connection and complete setup
 
-5. **Verify Connectivity**
-   - Use built-in connection verification tool
-   - Confirm **Director** successfully connects to **DataStream** platform
-   - Complete setup process once connectivity is established
+#### Navigation
+
+The wizard displays a 3-step progress indicator (Director Setup > Access Management > Connect Director). Click completed steps to navigate back. Invalid steps are marked with an error indicator.
+
+**Navigation Buttons:**
+- <gui>Cancel</gui> - Exit wizard (changes to <gui>Connect Later</gui> on final step)
+- <gui>Next</gui> - Advance to next step with validation
+- <gui>Create Director</gui> - Submit Director configuration and proceed to connection (from Step 2)
+- <gui>Complete Setup</gui> - Finalize Director creation (final step, enabled after successful connection)
 
 ### Network Requirements
 
@@ -241,9 +253,7 @@ If your firewall processes rules in order, ensure the `dl.vget.me` rule is evalu
 
 ### Security Considerations
 
-Deploy **Directors** in appropriate network segments and implement network access controls and monitoring. Use dedicated service accounts with minimal privileges and enable logging and audit trails for security monitoring.
-
-All sensitive data processing occurs locally on the **Director**, with only configuration metadata transmitted to the cloud platform. Implement encryption for data at rest and in transit, and maintain regular security updates and patch management.
+Deploy **Directors** in appropriate network segments and implement network access controls and monitoring. Use dedicated service accounts with minimal privileges and enable logging and audit trails for security monitoring. Implement encryption for data at rest and in transit, and maintain regular security updates and patch management.
 
 ### Performance Optimization
 
@@ -255,7 +265,7 @@ For data processing efficiency, optimize YAML pipeline configurations for perfor
 
 Maintain regular configuration backups and version control. Document recovery procedures and test them regularly. Implement monitoring and alerting for service health and plan for disaster recovery scenarios.
 
-For redundancy, group **Directors** into clusters for automatic failover and load balancing. Clusters require a minimum of 3 Directors (odd number for quorum) and provide continuous operation when individual Directors fail. See <Topic id="directors-clusters">Clusters</Topic> for configuration details. Consider geographic distribution for disaster recovery and plan for seamless failover procedures.
+For redundancy, group **Directors** into clusters for automatic failover and load balancing. See <Topic id="directors-clusters">Clusters</Topic> for configuration details and quorum requirements. Consider geographic distribution for disaster recovery and plan for seamless failover procedures.
 
 ## Troubleshooting