Add configuration options to adjust log level and log privacy level on non-Apple platforms

Also adjust the log privacy level on non-Apple platforms to `public` and don’t log potentially sensitive information by default.

rdar://132525691
Resolves #1591

Author: ahoppen

CONTRIBUTING.md

@@ -98,14 +98,7 @@ Or to stream the logs as they are produced:
 log stream --predicate 'subsystem CONTAINS "org.swift.sourcekit-lsp"'  --level debug
 ```
 
-SourceKit-LSP masks data that may contain private information such as source file names and contents by default. To enable logging of this information, run
-
-```sh
-sudo log config --subsystem org.swift.sourcekit-lsp --mode private_data:on
-```
-
-To enable more verbose logging on non-macOS platforms, launch sourcekit-lsp with the `SOURCEKITLSP_LOG_LEVEL` environment variable set to `debug`.
-
+SourceKit-LSP masks data that may contain private information such as source file names and contents by default. To enable logging of this information, follow the instructions in [Diagnose Bundle.md](Documentation/Diagnose%20Bundle.md).
 
 ## Formatting
 

Documentation/Configuration File.md

@@ -36,6 +36,9 @@ The structure of the file is currently not guaranteed to be stable. Options may
     - `indexPrefixMap: [string: string]`: Path remappings for remapping index data for local use.
     - `maxCoresPercentageToUseForBackgroundIndexing: double`: A hint indicating how many cores background indexing should use at most (value between 0 and 1). Background indexing is not required to honor this setting
     - `updateIndexStoreTimeout: int`: Number of seconds to wait for an update index store task to finish before killing it.
+- `logging`: Dictionary with the following keys, changing SourceKit-LSP’s logging behavior on non-Apple platforms. On Apple platforms, logging is done through the [system log](Diagnose%20Bundle.md#Enable%20Extended%20Logging). These options can only be set globally and not per workspace.
+  - `logLevel: "debug"|"info"|"default"|"error"|"fault"`: The level from which one onwards log messages should be written.
+  - `privacyLevel: "public"|"private"|"sensitive"`: Whether potentially sensitive information should be redacted. Default is `public`, which redacts potentially sensitive information.
 - `defaultWorkspaceType: "buildserver"|"compdb"|"swiftpm"`: Overrides workspace type selection logic.
 - `generatedFilesPath: string`: Directory in which generated interfaces and macro expansions should be stored.
 - `backgroundIndexing: bool`: Explicitly enable or disable background indexing.

Documentation/Diagnose Bundle.md

@@ -20,6 +20,20 @@ The diagnose bundle contains the following information:
 
 Extended logging of SourceKit-LSP is not enabled by default because it contains information about your source code, directory structure and similar potentially sensitive information. Instead, the logging system redacts that information. If you are comfortable with sharing such information, you can enable extended SourceKit-LSP’s extended logging, which improves the ability of SourceKit-LSP developers to understand and fix issues.
 
+### macOS
+
 To enable extended logging on macOS, install the configuration profile from https://github.com/swiftlang/sourcekit-lsp/blob/main/Documentation/Enable%20Extended%20Logging.mobileconfig as described in https://support.apple.com/guide/mac-help/configuration-profiles-standardize-settings-mh35561/mac#mchlp41bd550. SourceKit-LSP will immediately stop redacting information and include them in the system log.
 
 To disable extended logging again, remove the configuration profile as described in https://support.apple.com/guide/mac-help/configuration-profiles-standardize-settings-mh35561/mac#mchlpa04df41.
+
+### Non-Apple platforms
+
+Create a [configuration file](Configuration%20File.md) with the following contents at `~/.sourcekit-lsp/config.json` with the following contents:
+```json
+{
+  "logging": {
+    "level": "debug",
+    "privacyLevel": "private"
+  }
+}
+```

Documentation/Environment Variables.md

@@ -10,8 +10,8 @@ The following environment variables can be used to control some behavior in Sour
 
 ## Runtime
 
-- `SOURCEKITLSP_LOG_LEVEL`: When using `NonDarwinLogger`, specify the level at which messages should be logged. Defaults to `default`. Use `debug` to increase to the highest log level.
-- `SOURCEKITLSP_LOG_PRIVACY_LEVEL`: When using `NonDarwinLogger`, specifies whether information that might contain personal information (essentially source code) should be logged. Defaults to `private`, which logs this information. Set to `public` to redact this information.
+- `SOURCEKITLSP_LOG_LEVEL`: When using `NonDarwinLogger`, specify the level at which messages should be logged. Defaults to `debug` in debug build and `default` in release builds. Primarily used to increase the log level when running tests from a release build in Swift CI. To adjust the logging on user devices, use the [Configuration file](Configuration%20File.md).
+- `SOURCEKITLSP_LOG_PRIVACY_LEVEL`: When using `NonDarwinLogger`, specifies whether information that might contain sensitive information (essentially source code) should be logged. Defaults to `private` in debug build and `public` in release builds. Primarily used to log sensitive information when running tests from a release build in Swift CI. To adjust the logging on user devices, use the [Configuration file](Configuration%20File.md).
 
 ## Testing
 - `SKIP_LONG_TESTS`: Skip tests that typically take more than 1s to execute.

Sources/SKLogging/NonDarwinLogging.swift

@@ -23,26 +23,36 @@ import Foundation
 
 package enum LogConfig {
   /// The globally set log level
-  fileprivate static let logLevel: NonDarwinLogLevel = {
-    if let envVar = ProcessInfo.processInfo.environment["SOURCEKITLSP_LOG_LEVEL"],
-      let logLevel = NonDarwinLogLevel(envVar)
-    {
-      return logLevel
-    }
-    #if DEBUG
-    return .debug
-    #else
-    return .default
-    #endif
-  }()
+  package static let logLevel = ThreadSafeBox<NonDarwinLogLevel>(
+    initialValue: {
+      if let envVar = ProcessInfo.processInfo.environment["SOURCEKITLSP_LOG_LEVEL"],
+        let logLevel = NonDarwinLogLevel(envVar)
+      {
+        return logLevel
+      }
+      #if DEBUG
+      return .debug
+      #else
+      return .default
+      #endif
+    }()
+  )
 
   /// The globally set privacy level
-  fileprivate static let privacyLevel: NonDarwinLogPrivacy = {
-    guard let envVar = ProcessInfo.processInfo.environment["SOURCEKITLSP_LOG_PRIVACY_LEVEL"] else {
+  package static let privacyLevel = ThreadSafeBox<NonDarwinLogPrivacy>(
+    initialValue: {
+      if let envVar = ProcessInfo.processInfo.environment["SOURCEKITLSP_LOG_PRIVACY_LEVEL"],
+        let privacyLevel = NonDarwinLogPrivacy(envVar)
+      {
+        return privacyLevel
+      }
+      #if DEBUG
       return .private
-    }
-    return NonDarwinLogPrivacy(envVar) ?? .private
-  }()
+      #else
+      return .public
+      #endif
+    }()
+  )
 }
 
 /// A type that is API-compatible to `OSLogType` for all uses within
@@ -114,22 +124,13 @@ package enum NonDarwinLogPrivacy: Comparable, Sendable {
   case `private`
   case sensitive
 
-  init?(_ value: String) {
+  package init?(_ value: String) {
     switch value.lowercased() {
     case "sensitive": self = .sensitive
     case "private": self = .private
     case "public": self = .public
-    default: break
-    }
-
-    switch Int(value) {
-    case 0: self = .public
-    case 1: self = .private
-    case 2: self = .sensitive
-    default: break
+    default: return nil
     }
-
-    return nil
   }
 }
 
@@ -314,8 +315,8 @@ package struct NonDarwinLogger: Sendable {
   ) {
     self.subsystem = subsystem
     self.category = category
-    self.logLevel = logLevel ?? LogConfig.logLevel
-    self.privacyLevel = privacyLevel ?? LogConfig.privacyLevel
+    self.logLevel = logLevel ?? LogConfig.logLevel.value
+    self.privacyLevel = privacyLevel ?? LogConfig.privacyLevel.value
     self.overrideLogHandler = overrideLogHandler
   }
 

Sources/SKOptions/SourceKitLSPOptions.swift

@@ -210,6 +210,36 @@ public struct SourceKitLSPOptions: Sendable, Codable, CustomLogStringConvertible
     }
   }
 
+  public struct LoggingOptions: Sendable, Codable, CustomLogStringConvertible {
+    /// The level from which one onwards log messages should be written.
+    public var level: String?
+    /// Whether potentially sensitive information should be redacted.
+    public var privacyLevel: String?
+
+    public init(
+      level: String? = nil,
+      privacyLevel: String? = nil
+    ) {
+      self.level = level
+      self.privacyLevel = privacyLevel
+    }
+
+    static func merging(base: LoggingOptions, override: LoggingOptions?) -> LoggingOptions {
+      return LoggingOptions(
+        level: override?.level ?? base.level,
+        privacyLevel: override?.privacyLevel ?? base.privacyLevel
+      )
+    }
+
+    public var description: String {
+      recursiveDescription(of: self)
+    }
+
+    public var redactedDescription: String {
+      recursiveRedactedDescription(of: self)
+    }
+  }
+
   public enum BackgroundPreparationMode: String {
     /// Build a target to prepare it
     case build
@@ -228,6 +258,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, CustomLogStringConvertible
   public var fallbackBuildSystem: FallbackBuildSystemOptions
   public var clangdOptions: [String]?
   public var index: IndexOptions
+  public var logging: LoggingOptions
 
   /// Default workspace type (buildserver|compdb|swiftpm). Overrides workspace type selection logic.
   public var defaultWorkspaceType: WorkspaceType?
@@ -301,6 +332,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, CustomLogStringConvertible
     compilationDatabase: CompilationDatabaseOptions = .init(),
     clangdOptions: [String]? = nil,
     index: IndexOptions = .init(),
+    logging: LoggingOptions = .init(),
     defaultWorkspaceType: WorkspaceType? = nil,
     generatedFilesPath: String? = nil,
     backgroundIndexing: Bool? = nil,
@@ -315,6 +347,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, CustomLogStringConvertible
     self.compilationDatabase = compilationDatabase
     self.clangdOptions = clangdOptions
     self.index = index
+    self.logging = logging
     self.generatedFilesPath = generatedFilesPath
     self.defaultWorkspaceType = defaultWorkspaceType
     self.backgroundIndexing = backgroundIndexing
@@ -368,6 +401,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, CustomLogStringConvertible
       ),
       clangdOptions: override?.clangdOptions ?? base.clangdOptions,
       index: IndexOptions.merging(base: base.index, override: override?.index),
+      logging: LoggingOptions.merging(base: base.logging, override: override?.logging),
       defaultWorkspaceType: override?.defaultWorkspaceType ?? base.defaultWorkspaceType,
       generatedFilesPath: override?.generatedFilesPath ?? base.generatedFilesPath,
       backgroundIndexing: override?.backgroundIndexing ?? base.backgroundIndexing,
@@ -405,6 +439,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, CustomLogStringConvertible
       try container.decodeIfPresent(FallbackBuildSystemOptions.self, forKey: CodingKeys.fallbackBuildSystem) ?? .init()
     self.clangdOptions = try container.decodeIfPresent([String].self, forKey: CodingKeys.clangdOptions)
     self.index = try container.decodeIfPresent(IndexOptions.self, forKey: CodingKeys.index) ?? .init()
+    self.logging = try container.decodeIfPresent(LoggingOptions.self, forKey: .logging) ?? .init()
     self.defaultWorkspaceType = try container.decodeIfPresent(
       WorkspaceType.self,
       forKey: CodingKeys.defaultWorkspaceType

Sources/sourcekit-lsp/SourceKitLSP.swift

@@ -276,6 +276,16 @@ struct SourceKitLSP: AsyncParsableCommand {
       fatalError("failed to redirect stdout -> stderr: \(strerror(errno)!)")
     }
 
+    let globalConfigurationOptions = globalConfigurationOptions
+    if let logLevelStr = globalConfigurationOptions.logging.level, let logLevel = NonDarwinLogLevel(logLevelStr) {
+      LogConfig.logLevel.value = logLevel
+    }
+    if let privacyLevelStr = globalConfigurationOptions.logging.privacyLevel,
+      let privacyLevel = NonDarwinLogPrivacy(privacyLevelStr)
+    {
+      LogConfig.privacyLevel.value = privacyLevel
+    }
+
     let realStdoutHandle = FileHandle(fileDescriptor: realStdout, closeOnDealloc: false)
 
     // Directory should match the directory we are searching for logs in `DiagnoseCommand.addNonDarwinLogs`.