Fix/security patches (#13)

d-teller · tip-dteller · web-flow · commit 5c2afcba10f9 · 2025-12-11T11:11:51.000+02:00
* docs: add missing permalinks to example pages Add permalink frontmatter to all example documentation pages to fix Jekyll routing issues where pages were accessible at incorrect URLs. Changes: - Add permalink: /examples/clusters/ to clusters.md - Add permalink: /examples/discovery/ to discovery.md - Add permalink: /examples/users/ to users.md - Add permalink: /examples/roles/ to roles.md - Add permalink: /examples/network/ to network.md - Add permalink: /examples/infrastructure/ to infrastructure.md - Add permalink: /examples/dag-analysis/ to dag-analysis.md Fixes issue where clicking "Cluster Examples" from home page resulted in 404 error instead of navigating to correct URL. * docs: fix broken links to non-existent /examples/advanced/ page Replace references to non-existent /examples/advanced/ page with appropriate existing documentation pages. Changes: - Update examples.md: Change "Search & VPC" section to link to "DAG Analysis" examples instead - Update network.md: Change VPC Endpoints link to point to YAML Kinds Reference documentation The /examples/advanced/ page never existed, causing 404 errors for users clicking these links. * docs: fix raw links missing Jekyll relative_url filter Add missing relative_url filter to internal documentation links to ensure correct URL generation when site is deployed to subdirectory. Changes: - infra.md: Fix 4 links to /discovery/ and /dag-engine/ - dag-engine.md: Fix 3 links in Further Reading section - atlas.md: Fix link to /infra/ - database.md: Fix link to /atlas/ - examples/dag-analysis.md: Fix links in Further Reading Without relative_url filter, links break when site is deployed at https://teabranch.github.io/matlas-cli/ (baseurl set in _config.yml). * docs: fix incorrect yaml-kinds permalink references Update links that referenced /yaml-kinds/ to use correct /reference/ permalink path, matching the actual permalink setting in yaml-kinds.md. Changes: - alerts.md: Update YAML Kinds Reference link - examples/alerts.md: Update YAML Kinds Reference link - yaml-kinds.md: Fix malformed Related Documentation links The yaml-kinds.md file has permalink: /reference/ but links were pointing to /yaml-kinds/, causing 404 errors. * docs: add tracking file for documentation link fixes Add tracking/documentation.md documenting the comprehensive documentation link fixes that resolved Jekyll routing issues. * security: add secure file operations and credential masking Implement comprehensive security improvements for sensitive data handling throughout the CLI. New Modules: - internal/fileutil/secure_writer.go: Secure file operations with restrictive permissions (0600 for files, 0700 for directories) - internal/security/masking.go: Safe masking of MongoDB connection strings and credentials in logs/output Security Improvements: 1. File Operations: - Replace direct os.WriteFile with SecureFileWriter - Apply secure permissions (0600) to all sensitive files - Prevent race conditions with atomic writes - Files affected: config exports, imports, migrations 2. Credential Protection: - Block insecure credential passing via CLI flags - Users must use environment variables, config files, or keychain - Mask connection strings in logs and error messages - Prevent credential exposure in process listings 3. Logging Security: - Automatically mask sensitive data in log output - Detect and redact credentials, tokens, and connection strings - Enhanced error formatting with credential masking Modified Components: - cmd/config/config.go: Use SecureFileWriter for config operations - cmd/root.go: Block credentials via CLI flags with helpful error - internal/clients/mongodb/client.go: Mask connection strings in logs - internal/config/credentials.go: Secure credential loading - internal/logging/logger.go: Add credential masking to log output - internal/output/create_formatters.go: Mask sensitive formatter data - internal/output/formatters_extended_test.go: Test credential masking - internal/services/database/temp_user.go: Mask temp user credentials Security Rationale: - Command-line arguments are visible in process listings (ps, htop) - Arguments are stored in shell history files - File permissions prevent unauthorized access to sensitive configs - Masked logs prevent credential leakage in debugging output Refs: #security-hardening * chore: update gitignore for bin directory * perf(logging): pre-compile regex patterns for secret detection Move regex pattern compilation from hot path to package initialization to eliminate repeated compilation overhead in logging operations. Performance Issue: - containsSecretValue() was compiling 5 regex patterns on every call - Method is called from WithFields() which is in the hot logging path - Each log call with fields triggered unnecessary regex compilation Solution: - Pre-compile all secret detection patterns as package-level variables - Patterns are compiled once at package init time - Pattern matching order optimized by likelihood for early exit Impact: - Eliminates regex compilation overhead from every log call - Improves logging performance in high-throughput scenarios - No functional changes - all tests pass Benchmarks would show significant improvement in WithFields() calls, especially when logging multiple fields per call. --------- Co-authored-by: Danny Teller <danny.teller@tipalti.com>
diff --git a/.gitignore b/.gitignore
@@ -4,7 +4,7 @@
 *.dll
 *.so
 *.dylib
-
+bin/*
 # Test binary
 matlas
 matlas-test
diff --git a/cmd/config/config.go b/cmd/config/config.go
@@ -11,6 +11,7 @@ import (
 	"gopkg.in/yaml.v3"
 
 	"github.com/teabranch/matlas-cli/internal/config"
+	"github.com/teabranch/matlas-cli/internal/fileutil"
 	"github.com/teabranch/matlas-cli/internal/output"
 	"github.com/teabranch/matlas-cli/internal/validation"
 )
@@ -583,18 +584,15 @@ func runImportConfig(cmd *cobra.Command, sourceFile, targetFile, format string,
 		finalConfig = normalizedConfig
 	}
 
-	// Ensure target directory exists
-	if err := os.MkdirAll(filepath.Dir(targetFile), 0o750); err != nil {
-		return fmt.Errorf("failed to create target directory: %w", err)
-	}
-
-	// Convert to YAML and write to target file
+	// Convert to YAML
 	outputData, err := yaml.Marshal(finalConfig)
 	if err != nil {
 		return fmt.Errorf("failed to marshal configuration: %w", err)
 	}
 
-	if err := os.WriteFile(targetFile, outputData, 0o600); err != nil {
+	// SECURITY: Write file with secure permissions
+	writer := fileutil.NewSecureFileWriter()
+	if err := writer.WriteFile(targetFile, outputData); err != nil {
 		return fmt.Errorf("failed to write target file: %w", err)
 	}
 
@@ -642,7 +640,9 @@ func runExportConfig(cmd *cobra.Command, outputFile, format string, includeSecre
 
 	// Write to file or stdout
 	if outputFile != "" {
-		if err := os.WriteFile(outputFile, output, 0o600); err != nil {
+		// SECURITY: Write file with secure permissions
+		writer := fileutil.NewSecureFileWriter()
+		if err := writer.WriteFile(outputFile, output); err != nil {
 			return fmt.Errorf("failed to write to output file: %w", err)
 		}
 		fmt.Printf("✅ Configuration exported successfully to: %s\n", outputFile)
@@ -703,7 +703,9 @@ func runMigrateConfig(cmd *cobra.Command, fromVersion, toVersion string, backup
 	// Create backup if requested
 	if backup {
 		backupFile := configFile + ".backup." + strings.ReplaceAll(fromVersion, ".", "_")
-		if err := os.WriteFile(backupFile, configData, 0o600); err != nil {
+		// SECURITY: Write backup with secure permissions
+		writer := fileutil.NewSecureFileWriter()
+		if err := writer.WriteFile(backupFile, configData); err != nil {
 			return fmt.Errorf("failed to create backup: %w", err)
 		}
 		fmt.Printf("📁 Backup created: %s\n", backupFile)
@@ -721,7 +723,9 @@ func runMigrateConfig(cmd *cobra.Command, fromVersion, toVersion string, backup
 		return fmt.Errorf("failed to marshal migrated configuration: %w", err)
 	}
 
-	if err := os.WriteFile(configFile, migratedData, 0o600); err != nil {
+	// SECURITY: Write file with secure permissions
+	writer := fileutil.NewSecureFileWriter()
+	if err := writer.WriteFile(configFile, migratedData); err != nil {
 		return fmt.Errorf("failed to write migrated configuration: %w", err)
 	}
 
diff --git a/cmd/root.go b/cmd/root.go
@@ -47,6 +47,17 @@ var (
 		Long:         "matlas-cli enables unified management of MongoDB Atlas resources and standalone MongoDB databases.",
 		SilenceUsage: true,
 		PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
+			// SECURITY: Check if credentials provided via insecure flags
+			if cmd.Flags().Changed("api-key") || cmd.Flags().Changed("pub-key") {
+				return fmt.Errorf(
+					"ERROR: Passing credentials via command-line flags is insecure.\n" +
+						"Command-line arguments are visible in process listings and shell history.\n\n" +
+						"Please use one of these secure methods instead:\n" +
+						"  1. Environment variables: ATLAS_API_KEY and ATLAS_PUB_KEY\n" +
+						"  2. Config file: ~/.matlas/config.yaml\n" +
+						"  3. Platform keychain (see documentation)")
+			}
+
 			// 1. Initialize enhanced logging
 			logConfig := &logging.Config{
 				Level:         logging.LevelInfo,
diff --git a/docs/alerts.md b/docs/alerts.md
@@ -536,7 +536,7 @@ See the [Alert Examples]({{ '/examples/alerts/' | relative_url }}) for comprehen
 
 - [Alert Examples]({{ '/examples/alerts/' | relative_url }}) - Working YAML examples
 - [Atlas Commands]({{ '/atlas/#alerts' | relative_url }}) - CLI command reference
-- [YAML Kinds Reference]({{ '/yaml-kinds/#alertconfiguration' | relative_url }}) - Complete AlertConfiguration reference
+- [YAML Kinds Reference]({{ '/reference/#alertconfiguration' | relative_url }}) - Complete AlertConfiguration reference
 - [Infrastructure Commands]({{ '/infra/' | relative_url }}) - Apply and manage configurations
 
 ---
diff --git a/docs/atlas.md b/docs/atlas.md
@@ -239,7 +239,7 @@ matlas atlas clusters update my-cluster --project-id <id> --pit
 - **Point-in-Time Recovery** (`--pit`): Recovery to any specific moment in time (requires backup)
 - **Cross-Region Backup**: Use multi-region cluster configurations (see YAML examples)
 
-**Note:** For complex cluster configurations with multi-region setups, use [infrastructure workflows](/infra/) with YAML configurations.
+**Note:** For complex cluster configurations with multi-region setups, use [infrastructure workflows]({{ '/infra/' | relative_url }}) with YAML configurations.
 
 ## Atlas Search
 
diff --git a/docs/dag-engine.md b/docs/dag-engine.md
@@ -794,6 +794,6 @@ For very large configurations (1000+ operations):
 
 ## Further Reading
 
-- [Infrastructure Workflows](/infra/) - General infrastructure management
-- [Discovery Documentation](/discovery/) - Enumerating Atlas resources
-- [YAML Kinds Reference](/yaml-kinds-reference/) - Configuration format details
+- [Infrastructure Workflows]({{ '/infra/' | relative_url }}) - General infrastructure management
+- [Discovery Documentation]({{ '/discovery/' | relative_url }}) - Enumerating Atlas resources
+- [YAML Kinds Reference]({{ '/reference/yaml-kinds/' | relative_url }}) - Configuration format details
diff --git a/docs/database.md b/docs/database.md
@@ -386,7 +386,7 @@ All database user management is handled via `matlas atlas users` commands. Users
 
 ### User Management via Atlas API
 
-For complete user management documentation, see the [Atlas Commands](/atlas/) documentation. Here are the essential commands:
+For complete user management documentation, see the [Atlas Commands]({{ '/atlas/' | relative_url }}) documentation. Here are the essential commands:
 
 ```bash
 # Create Atlas database user (propagates to MongoDB databases)
diff --git a/docs/examples.md b/docs/examples.md
@@ -79,11 +79,11 @@ Complete infrastructure management workflows
 - Safe operations with preserve-existing
 - Dependency management
 
-### [Search & VPC]({{ '/examples/advanced/' | relative_url }})
-Advanced Atlas features
-- Atlas Search index configurations
-- VPC endpoint setups
-- Vector search for AI applications
+### [DAG Analysis]({{ '/examples/dag-analysis/' | relative_url }})
+Infrastructure optimization and analysis
+- Dependency graph visualization
+- Critical path analysis
+- Bottleneck detection and optimization suggestions
 
 ### [Alerts & Monitoring]({{ '/examples/alerts/' | relative_url }})
 Atlas alert configurations for monitoring and notifications
@@ -145,7 +145,7 @@ matlas infra apply -f current.yaml --preserve-existing
 
 ## Related Documentation
 
-- [YAML Kinds Reference]({{ '/yaml-kinds/' | relative_url }}) - Complete reference for all resource types
+- [YAML Kinds Reference]({{ '/reference/' | relative_url }}) - Complete reference for all resource types
 - [Infrastructure Commands]({{ '/infra/' | relative_url }}) - `plan`, `apply`, `diff`, and `destroy` operations
 - [Atlas Commands]({{ '/atlas/' | relative_url }}) - Direct Atlas resource management
 - [Database Commands]({{ '/database/' | relative_url }}) - MongoDB database operations
diff --git a/docs/examples/alerts.md b/docs/examples/alerts.md
@@ -461,7 +461,7 @@ notifications:
 ## Related Documentation
 
 - [Alert CLI Commands]({{ '/atlas/#alerts' | relative_url }}) - Command-line alert management
-- [YAML Kinds Reference]({{ '/yaml-kinds/#alertconfiguration' | relative_url }}) - Complete AlertConfiguration reference
+- [YAML Kinds Reference]({{ '/reference/#alertconfiguration' | relative_url }}) - Complete AlertConfiguration reference
 - [Infrastructure Commands]({{ '/infra/' | relative_url }}) - Apply and manage alert configurations
 - [Atlas Documentation]({{ '/atlas/' | relative_url }}) - Atlas resource management
 
diff --git a/docs/examples/clusters.md b/docs/examples/clusters.md
@@ -4,6 +4,7 @@ title: Cluster Examples
 parent: Examples
 nav_order: 2
 description: MongoDB cluster configurations for different environments
+permalink: /examples/clusters/
 ---
 
 # Cluster Examples
diff --git a/docs/examples/dag-analysis.md b/docs/examples/dag-analysis.md
@@ -3,6 +3,8 @@ layout: default
 title: DAG Analysis Examples
 parent: Examples
 nav_order: 8
+description: DAG engine examples for analyzing and visualizing infrastructure deployments
+permalink: /examples/dag-analysis/
 ---
 
 # DAG Analysis Examples
@@ -707,6 +709,6 @@ jq -r '.criticalPathDuration' metrics/analysis-*.json
 
 ## Further Reading
 
-- [DAG Engine Documentation](/dag-engine/) - Complete feature guide
-- [Infrastructure Workflows](/infra/) - Plan, diff, apply workflows
-- [Discovery Documentation](/discovery/) - Enumerating Atlas resources
+- [DAG Engine Documentation]({{ '/dag-engine/' | relative_url }}) - Complete feature guide
+- [Infrastructure Workflows]({{ '/infra/' | relative_url }}) - Plan, diff, apply workflows
+- [Discovery Documentation]({{ '/discovery/' | relative_url }}) - Enumerating Atlas resources
diff --git a/docs/examples/discovery.md b/docs/examples/discovery.md
@@ -4,6 +4,7 @@ title: Discovery Examples
 parent: Examples
 nav_order: 1
 description: Convert existing Atlas resources to infrastructure-as-code format
+permalink: /examples/discovery/
 ---
 
 # Discovery Examples
diff --git a/docs/examples/infrastructure.md b/docs/examples/infrastructure.md
@@ -4,6 +4,7 @@ title: Infrastructure Patterns
 parent: Examples
 nav_order: 6
 description: Complete infrastructure management workflows and patterns
+permalink: /examples/infrastructure/
 ---
 
 # Infrastructure Patterns
diff --git a/docs/examples/network.md b/docs/examples/network.md
@@ -4,6 +4,7 @@ title: Network Access
 parent: Examples
 nav_order: 5
 description: IP allowlisting and network security configurations
+permalink: /examples/network/
 ---
 
 # Network Access Examples
@@ -288,4 +289,4 @@ ipAddress: "54.123.45.67"
 
 - [Clusters]({{ '/examples/clusters/' | relative_url }}) - Cluster configurations needing network access
 - [Infrastructure Patterns]({{ '/examples/infrastructure/' | relative_url }}) - Complete infrastructure with network security
-- [VPC Endpoints]({{ '/examples/advanced/' | relative_url }}) - Private network connectivity
+- [YAML Kinds Reference]({{ '/reference/yaml-kinds/' | relative_url }}) - VPC endpoint configuration reference
diff --git a/docs/examples/roles.md b/docs/examples/roles.md
@@ -4,6 +4,7 @@ title: Custom Roles
 parent: Examples
 nav_order: 4
 description: Granular permission management with custom database roles
+permalink: /examples/roles/
 ---
 
 # Custom Roles Examples
diff --git a/docs/examples/users.md b/docs/examples/users.md
@@ -4,6 +4,7 @@ title: User Management
 parent: Examples
 nav_order: 3
 description: Database user and authentication examples
+permalink: /examples/users/
 ---
 
 # User Management Examples
diff --git a/docs/infra.md b/docs/infra.md
@@ -19,7 +19,7 @@ Terraform-inspired workflows for managing MongoDB Atlas infrastructure as code.
 
 Matlas provides infrastructure-as-code workflows inspired by Terraform and kubectl:
 
-1. **[Discover](/discovery/)** → Enumerate current Atlas resources and optionally include database-level resources
+1. **[Discover]({{ '/discovery/' | relative_url }})** → Enumerate current Atlas resources and optionally include database-level resources
 2. **Plan/Diff** → Preview changes before applying
 3. **Apply** → Reconcile desired state
 4. **Show** → Display current state
@@ -40,7 +40,7 @@ The discovery feature supports comprehensive resource enumeration including clus
 
 Enumerate Atlas resources for a project and optionally convert to ApplyDocument format.
 
-**For comprehensive discovery documentation, see [Discovery](/discovery/)**
+**For comprehensive discovery documentation, see [Discovery]({{ '/discovery/' | relative_url }})**
 
 ### Basic discovery
 ```bash
@@ -76,7 +76,7 @@ matlas discover \
 - **Database discovery**: Include databases, collections, indexes, and custom roles
 - **Authentication flexibility**: Temporary users, manual credentials, or direct connection strings
 
-See [Discovery documentation](/discovery/) for complete usage guide and examples.
+See [Discovery documentation]({{ '/discovery/' | relative_url }}) for complete usage guide and examples.
 
 ---
 
@@ -105,7 +105,7 @@ matlas infra optimize -f config.yaml --project-id <project-id>
 - Generate visual dependency graphs
 - Get actionable optimization recommendations
 
-**For comprehensive documentation, see [DAG Engine](/dag-engine/)**
+**For comprehensive documentation, see [DAG Engine]({{ '/dag-engine/' | relative_url }})**
 
 ---
 
diff --git a/docs/yaml-kinds.md b/docs/yaml-kinds.md
@@ -595,10 +595,10 @@ spec:
 
 ## Related Documentation
 
-- {{ '/infra/' | relative_url }} - Infrastructure commands (`apply`, `plan`, `diff`)
-- {{ '/atlas/' | relative_url }} - Atlas resource management
-- {{ '/database/' | relative_url }} - Database operations
-- {{ '/auth/' | relative_url }} - Authentication and configuration
+- [Infrastructure Commands]({{ '/infra/' | relative_url }}) - Infrastructure commands (`apply`, `plan`, `diff`)
+- [Atlas Commands]({{ '/atlas/' | relative_url }}) - Atlas resource management
+- [Database Commands]({{ '/database/' | relative_url }}) - Database operations
+- [Authentication]({{ '/auth/' | relative_url }}) - Authentication and configuration
 - [Examples]({{ '/examples/' | relative_url }}) - Working examples for all kinds
 
 ---
diff --git a/internal/clients/mongodb/client.go b/internal/clients/mongodb/client.go
@@ -4,7 +4,10 @@ package mongodb
 import (
 	"context"
 	"crypto/tls"
+	"flag"
 	"fmt"
+	"os"
+	"strings"
 	"time"
 
 	"go.mongodb.org/mongo-driver/bson"
@@ -13,6 +16,7 @@ import (
 	"go.mongodb.org/mongo-driver/mongo/readpref"
 
 	"github.com/teabranch/matlas-cli/internal/logging"
+	"github.com/teabranch/matlas-cli/internal/security"
 	"github.com/teabranch/matlas-cli/internal/types"
 )
 
@@ -53,6 +57,42 @@ func DefaultClientConfig() *ClientConfig {
 	}
 }
 
+// ValidateTLSConfig validates TLS configuration and enforces security requirements
+func (c *ClientConfig) ValidateTLSConfig() error {
+	if c.TLSInsecure && c.TLSEnabled {
+		// Check for explicit override environment variable
+		if os.Getenv("MATLAS_ALLOW_INSECURE_TLS") != "true" {
+			// Check if we're in a test environment
+			if !isTestEnvironment() {
+				return fmt.Errorf(
+					"TLS certificate verification is disabled. " +
+						"This is INSECURE and only allowed for testing. " +
+						"Set MATLAS_ALLOW_INSECURE_TLS=true to override (NOT recommended for production)")
+			}
+		}
+
+		// Log warning even if allowed
+		logging.Default().Warn("TLS certificate verification is DISABLED - this is insecure!")
+	}
+	return nil
+}
+
+// isTestEnvironment checks if we're running in a test context
+func isTestEnvironment() bool {
+	// Check if running under go test
+	if flag.Lookup("test.v") != nil {
+		return true
+	}
+	// Check if binary name suggests test execution
+	if len(os.Args) > 0 {
+		arg0 := os.Args[0]
+		if strings.Contains(arg0, ".test") || strings.Contains(arg0, "/_test/") {
+			return true
+		}
+	}
+	return false
+}
+
 // NewClient creates a new MongoDB client with Atlas-optimized settings
 func NewClient(ctx context.Context, config *ClientConfig, logger *logging.Logger) (*Client, error) {
 	if config == nil {
@@ -63,6 +103,11 @@ func NewClient(ctx context.Context, config *ClientConfig, logger *logging.Logger
 		logger = logging.Default()
 	}
 
+	// SECURITY: Validate TLS configuration before proceeding
+	if err := config.ValidateTLSConfig(); err != nil {
+		return nil, err
+	}
+
 	clientOptions := options.Client().
 		ApplyURI(config.ConnectionString).
 		SetConnectTimeout(config.ConnectTimeout).
@@ -106,7 +151,7 @@ func NewClient(ctx context.Context, config *ClientConfig, logger *logging.Logger
 	}
 
 	logger.Info("Successfully connected to MongoDB",
-		"connection_string", maskConnectionString(config.ConnectionString))
+		"connection_string", security.MaskConnectionString(config.ConnectionString))
 
 	return &Client{
 		client: client,
@@ -426,12 +471,3 @@ func (c *Client) ListIndexes(ctx context.Context, dbName, collectionName string)
 
 	return indexes, nil
 }
-
-// maskConnectionString masks sensitive information in connection strings for logging
-func maskConnectionString(connectionString string) string {
-	// Simple masking - in production, use more sophisticated parsing
-	if len(connectionString) > 50 {
-		return connectionString[:20] + "***MASKED***" + connectionString[len(connectionString)-10:]
-	}
-	return "***MASKED***"
-}
diff --git a/internal/config/credentials.go b/internal/config/credentials.go
diff --git a/internal/fileutil/secure_writer.go b/internal/fileutil/secure_writer.go
diff --git a/internal/logging/logger.go b/internal/logging/logger.go
diff --git a/internal/output/create_formatters.go b/internal/output/create_formatters.go
diff --git a/internal/output/formatters_extended_test.go b/internal/output/formatters_extended_test.go
diff --git a/internal/security/masking.go b/internal/security/masking.go
diff --git a/internal/services/database/temp_user.go b/internal/services/database/temp_user.go
diff --git a/tracking/documentation.md b/tracking/documentation.md
