Report types "source" and "summary" appear to be swapped

[rosasurfer]

This has been a long-standing source of confusion for us and repeatedly comes up whenever developers switch from IDE-integrated PHPCS to command-line usage.

The current behavior appears counter-intuitive:
--report=source produces what most developers would expect to be a summary of issues.
--report=summary produces what most developers would expect to be a per-source (per-file) breakdown.

In practice, developers regularly assume the opposite based on the report names. The mismatch is confusing enough that we ended up adding a small shell wrapper in our projects which swaps the arguments to match expectations.

I was unable to find an existing issue about this, so I'm opening one here. With the new maintainer and the recent activity giving the project fresh momentum, this might be a good opportunity to revisit this long-standing naming inconsistency.

Proposed solution
Changing the current behavior directly would be BC breaking and therefor would require a new major release. Deferring this to v5 doesn't seem practical. As a non-breaking fix for v4.x, I'd propose introducing an opt-in runtime argument (and equivalent XML setting), for example:

--report-names=legacy|fixed

• legacy: the current behavior (default).
• fixed: swap the internal mapping so that source and summary match their intuitive meaning.

This would allow projects to migrate gradually without breaking CI setups or tooling. In v5, the default could then be switched to fixed, and the legacy mode eventually be deprecated.

edit: I'm willing to work on a PR if this update is appreciated.

[jrfnl]

@rosasurfer Thank you for opening this issue and bringing this to my attention.

While I understand your point, as far as I can see, this confusion has never been reported before by anyone else. Not in this repo, not in the old repo, not in the PEAR repo which preceded that.

As you correctly state, changing this would be a severe BC-break. I can only justify such a BC-break if it would unblock/reduce friction for a significantly large group of users, as otherwise we'd be solving the problem for a few, while creating a new problem for the many.

So before I even entertain the idea, I'd like to see whether this is an issue isolated to your team/company (possibly caused by some other tooling used by the team using the opposite meaning of the names), or whether more users of CodeSniffer are experiencing this confusion.

So, I'm going to leave this issue open for a while to allow others to add their voice to it.
Please do not ask your own team to spam the issue as we already know their opinion.

[fredden]

The current report names make sense to me.

summary provides a summary of the code-base, with each (problematic) file listed with how many problems (errors and warnings) it has.

source shows which coding standard rules (sniffs) are the "source" of these "problems".

I think the confusion may be one of perspective: thinking about the "source" being a coding standard rule (sniff) versus a violation of that rule (contents of a file); or a "summary" being a "summary of the rules which are broken" versus a "summary of the state of the code base".

I like the default report (--report=full) as this shows both the files and the violations in one go.

For reference, I have included the output of running with --report=summary and --report=source below so that those following along can see which is which.

Output of php bin/phpcs --standard=PSR12 --basepath=. --report=summary src/Util

PHP CODE SNIFFER REPORT SUMMARY
----------------------------------------------------------------------
FILE                                                  ERRORS  WARNINGS
----------------------------------------------------------------------
src/Util/Cache.php                                    3       2
src/Util/Common.php                                   3       2
src/Util/ExitCode.php                                 3       0
src/Util/Help.php                                     2       19
src/Util/IgnoreList.php                               2       0
src/Util/MessageCollector.php                         3       2
src/Util/Standards.php                                3       0
src/Util/Timing.php                                   2       0
src/Util/Tokens.php                                   2       0
src/Util/Writers/StatusWriter.php                     2       0
----------------------------------------------------------------------
A TOTAL OF 25 ERRORS AND 25 WARNINGS WERE FOUND IN 10 FILES
----------------------------------------------------------------------
PHPCBF CAN FIX 25 OF THESE SNIFF VIOLATIONS AUTOMATICALLY
----------------------------------------------------------------------
Time: 570ms; Memory: 12MB

Output of php bin/phpcs --standard=PSR12 --basepath=. --report=source src/Util

PHP CODE SNIFFER VIOLATION SOURCE SUMMARY
----------------------------------------------------------------------------------------
    STANDARD  CATEGORY            SNIFF                                            COUNT
----------------------------------------------------------------------------------------
[ ] Generic   Files               Line length too long                             25
[x] PSR12     Classes             Opening brace space found                        10
[x] PSR12     Files               File header spacing after tag block              10
[x] PSR12     Control structures  Control structure spacing first expression line  5
----------------------------------------------------------------------------------------
A TOTAL OF 50 SNIFF VIOLATIONS WERE FOUND IN 4 SOURCES
----------------------------------------------------------------------------------------
PHPCBF CAN FIX THE 3 MARKED SOURCES AUTOMATICALLY (25 VIOLATIONS IN TOTAL)
----------------------------------------------------------------------------------------
Time: 565ms; Memory: 12MB