JENKINS-75933 Allow to control logging of masking message

[vrenjith]

Ability to disable the logs that says "Masking" of credentials as this is filling up our logs.

Testing done

In progress

Submitter checklist

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira - https://issues.jenkins.io/browse/JENKINS-75933
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests that demonstrate the feature works or the issue is fixed

[jtnord]

as this is filling up our logs.

how is this filling up your logs? can you create an issue for this?

[jglick]

this is filling up our logs

is rather surprising. A single log line per withCredentials step noting which variables are due to be masked seems appropriate and necessary. In what scenario would this “fill up” logs?

[jglick]

https://javadoc.jenkins.io/jenkins/util/SystemProperties.html#getBoolean(java.lang.String,boolean) and please use positive rather than negative sense, e.g. enable mask logging on by default or set to false to override.

[vrenjith]

Okay @jglick
Let me change that.

[vrenjith]

Updated as per review comments @jglick

[vrenjith]

this is filling up our logs

is rather surprising. A single log line per withCredentials step noting which variables are due to be masked seems appropriate and necessary. In what scenario would this “fill up” logs?

Let me try to give you a sample log. I explained the scenario in https://issues.jenkins.io/browse/JENKINS-75933

[vrenjith]

this is filling up our logs

is rather surprising. A single log line per withCredentials step noting which variables are due to be masked seems appropriate and necessary. In what scenario would this “fill up” logs?

@jglick and @jtnord
When we look from a single usage of credentials closure, it looks quite okay. But when we talk about usage of many credentials (30+) which are again and again used in multiple steps, this makes up a lot of logs.

May be what is lacking here (as well as in the similar PR that I raised for the configuration plugin) is the understanding that this pipeline is a comprehensive pipeline that takes multiple products through the process and deploys them to 14 different datacenters.

Just to give a brief looks at the steps, see what happens just at the CI part alone.

                watch(config) { performI18NChecks(config) }
                watch(config) { performCustomPreBuildStep(config) }
                watch(config) { performGoGets(config) }
                watch(config) { performGoBuilds(config) }
                watch(config) { performMvnBuilds(config) }
                watch(config) { performNpmBuilds(config) }
                watch(config) { performMtaBuild(config) }
                watch(config) { performCustomBuildStep(config) }
                watch(config) { performRunKatalonTests(config) }
                watch(config) { performQmateTests(config, 0) }
                watch(config) { performPublishCoberturaResults(config) }
                watch(config) { performFortifyScan(config) }
                watch(config) { performGhasScan(config) }
                watch(config) { performWhiteSourceScan(config) }
                watch(config) { performCTPScan(config) }
                watch(config) { performCxOneScan(config) }
                watch(config) { performCheckmarXScan(config) }
                watch(config) { performPublishJacoco(config, config.collectContainerJacoco.size() == 0) }
                watch(config) { performSonarScanStep(config) }
                watch(config) { performPublishCheckstyle(config) }
                watch(config) { performPublishFindbugs(config) }
                watch(config) { performPublishOpenTasks(config) }
                watch(config) { performUploadFortifyResults(config) }
                watch(config) { performNpmPublish(config) }
                watch(config) { performDockerBuilds(config) }
                watch(config) { performVulnerabilityScan(config) }
                watch(config) { performCustomPostBuildStep(config) }
                watch(config) { performDescriptorValidation(config) }
                watch(config) { performManifestUpload(config) }
                watch(config) { performCustomUploadArtifacts(config) }

What I am sharing is just the CI part and this repeats for multipel products when an overall validation runs.
(I cannot share the entire log as such as that might violate the enterprise policies)

• Now, if we think about the number of credentials involved in this we can easily see how many logs will come one in the console logs.
• Same goes with the configuration files as every step uses a bunch of configuration files for its operation.

[jtnord]

watch(config) { performMvnBuilds(config) }

where are you using credentialsBinding? in watch(config) or performMvnBuilds(config)?
a maven build needs only a few credentials and the masking line entry is surely dwarfed by the maven output.

if you are doing this 100 times, still the line entries are hardly visible against the 100 of maven invocations, and would consume negligible disk space that necessitates their removal.

watch(config) { performPublishCheckstyle(config) }
watch(config) { performPublishFindbugs(config) }
watch(config) { performPublishOpenTasks(config) }

To where, Jenkins or sonarqube? if the latter why not performPublishSonarCube(), and if the former no credentials should be invloved here?

[vrenjith]

watch(config) { performMvnBuilds(config) }

where are you using credentialsBinding? in watch(config) or performMvnBuilds(config)? a maven build needs only a few credentials and the masking line entry is surely dwarfed by the maven output.

if you are doing this 100 times, still the line entries are hardly visible against the 100 of maven invocations, and would consume negligible disk space that necessitates their removal.

watch(config) { performPublishCheckstyle(config) }
watch(config) { performPublishFindbugs(config) }
watch(config) { performPublishOpenTasks(config) }

To where, Jenkins or sonarqube? if the latter why not performPublishSonarCube(), and if the former no credentials should be invloved here?

All those are closures where credentials are used. And it is also used multuple times for interacting with varuous systems.

[vrenjith]

watch(config) { performMvnBuilds(config) }

where are you using credentialsBinding? in watch(config) or performMvnBuilds(config)? a maven build needs only a few credentials and the masking line entry is surely dwarfed by the maven output.
if you are doing this 100 times, still the line entries are hardly visible against the 100 of maven invocations, and would consume negligible disk space that necessitates their removal.

watch(config) { performPublishCheckstyle(config) }
watch(config) { performPublishFindbugs(config) }
watch(config) { performPublishOpenTasks(config) }

To where, Jenkins or sonarqube? if the latter why not performPublishSonarCube(), and if the former no credentials should be invloved here?

All those are closures where credentials are used. And it is also used multuple times for interacting with varuous systems.

I would leave it here. As I am not sure what additional information I can give to have this approved.

[vrenjith]

Any comments?