[BUG][code-analyzer] Potential PMD Issue: exclude-pattern in Referenced Ruleset Not Respected

[ssaini01]

I'm encountering an issue with configuring code-analyzer to exclude files with specific prefixes. This is for a Salesforce org repository, and I'm using the PMD engine with a custom custom-ruleset.xml file. Despite specifying the prefixes to be ignored, the scanner still seems to process those files. For example I want to exclude Apex classes that have a prefix of fflib_ or TM. This is what the custom-ruleset.xml file looks like:

<?xml version="1.0" encoding="UTF-8"?>
<ruleset name="Custom Apex Rules"
         xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0
                             https://pmd.sourceforge.io/ruleset_2_0_0.xsd">

  <description>
    PMD ruleset that excludes fflib-related and TM-prefixed classes and their metadata files.
  </description>

  <!-- Exclude patterns -->
  <exclude-pattern>.*/fflib_.*\.(cls|trigger|cls-meta\.xml|trigger-meta\.xml)$</exclude-pattern>
  <exclude-pattern>.*/fflib-.*/.*\.(cls|trigger|cls-meta\.xml|trigger-meta\.xml)$</exclude-pattern>
  <exclude-pattern>.*/TM[^/]*\.(cls|trigger|cls-meta\.xml|trigger-meta\.xml)$</exclude-pattern>

  <!-- Apex rules -->
  <rule ref="category/apex/design.xml/CognitiveComplexity"/>
  <rule ref="category/apex/bestpractices.xml/ApexUnitTestClassShouldHaveAsserts"/>
</ruleset>

Do you have any suggestions on what I might be missing? Thanks in advance for your help!

[stephen-carter-at-sf]

Since you can select rules from multiple PMD ruleset files with the Code Analyzer rule selection mechanism, we don't directly give your ruleset file to PMD - but instead make it available to be referenced and then we generate a primary ruleset file that references the rules you have selected.

With this in mind, I wonder if PMD has a bug that if you reference a rule from a ruleset file - if it doesn’t actually pull in the exclusions for the rules in that ruleset file:
For example, I made a file called InnerRuleset.xml

<ruleset name="Custom Rules"
    xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">

    <description>
        My custom rules
    </description>

    <exclude-pattern>.*SomeClass.*</exclude-pattern>

    <rule ref="category/apex/codestyle.xml/ClassNamingConventions" />

</ruleset>

and applied it, and behind the scenes we rebuild the outermost ruleset based on your rule selection. When selecting just the one ClassNamingConventions, the new ruleset that we build references the other ruleset file… but I’m not sure if PMD applies the exclude-patterns in references or not. This is what we produce.

<?xml version="1.0" encoding="UTF-8"?>
<ruleset name="Ruleset for Salesforce Code Analyzer"
    xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
    <description>Rules to Run for Salesforce Code Analyzer</description>
    <rule ref="/Users/stephen.carter/temp/other/InnerRuleset.xml/ClassNamingConventions" />
</ruleset>

That’s my theory - I’ll test it out with PMD directly without code analyzer in the mix.

(Update):
My theory seems correct.
I created PMD Bug: pmd/pmd#5758

[nicolas1bruno]

@stephen-carter-at-sf As mentioned by @oowekyala in pmd/pmd#5758, it seems to be working as expected. Do you happen to have any workaround for this issue?

[adangel]

@ssaini01 Can you explain what "fflib-related and TM-prefixed classes" are and why you don't want to have these checked?

[adangel]

Oh, and do you want to have these files excluded for all rules or only the rules you have referenced (CognitiveComplexity and ApexUnitTestClassShouldHaveAsserts)?

[ssaini01]

@adangel - We were trying to exclude files that were part of library that my team does not manage. I was trying to exclude those files from all PMD checks.

[nicolas1bruno]

We have a similar situation where we need to exclude some files for only a few rules, but not for others. In Scanner version 4, we did this by running the analyzer multiple times, each with a different --pmdconfig flag to set different exclusions.
This behavior changed in version 5 and it’s blocking us from upgrading because now we can’t do that anymore.

[stephen-carter-at-sf]

Code Analyzer v5 does not use PMD Ruleset files as the list of rules to actually run, but instead allows users to provided PMD Ruleset files as a means of making rules available for selection (or modifying already available bundled rules).

Since the actual rules that run are based on the Code Analyzer's rule selector, we have to create an outer ruleset file with the list of rules actually selected. Unfortunately PMD does not forward the exclusions to our outer ruleset file.

With that said, Code Analyzer is missing its own exclusion/ignore abilities. This is on our radar and will be hopefully prioritized sometime this year.

For now, you'll have to simply provided multiple --target flags to only include the files you wish to include to scan.

[nicolas1bruno]

Having only this option makes the Code Analyzer VS Code extension impractical to use, as it leaves users unaware of the specific rules that should apply to each file.

[stephen-carter-at-sf]

Small update - we have started prioritizing work in this area and hope to soon within the next few months give an update - specifically regarding being able to ignore rule violations on specific files/folders. Thanks for your patience.