kie-issues#1670: Migrate Apache licenses check to apache rat 0.17.x#3189
kie-issues#1670: Migrate Apache licenses check to apache rat 0.17.x#3189jomarko wants to merge 1 commit intoapache:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR migrates the Apache RAT license header checking from version 0.16.1 to 0.17, addressing issue #1670. The update modifies how Apache RAT is invoked and how its output is parsed to detect license violations.
Key Changes:
- Upgraded Apache RAT from version 0.16.1 to 0.17
- Changed command-line arguments to use new syntax with
-Eflag and--separator - Implemented new output parsing logic with multiple grep patterns to detect unapproved licenses
- Enhanced error reporting with emoji indicators and formatted output extraction
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| java -jar apache-rat-0.16.1.jar -d . -E .rat-excludes | grep "== File:" && echo "The files listed above are missing license headers." && exit 1 || echo "All files have license headers." | ||
|
|
||
| java -jar apache-rat-0.17.jar -E .rat-excludes -- . > .rat-reports 2>&1 | ||
| if grep -q "^! Unapproved:" .rat-reports || grep -q "INFO: Unapproved: [1-9]" .rat-reports; then |
There was a problem hiding this comment.
The grep pattern "INFO: Unapproved: [1-9]" has a critical flaw: it requires exactly two spaces between "Unapproved:" and the digit. If Apache RAT 0.17 outputs a different number of spaces (e.g., single space or variable spacing based on number alignment), this pattern will fail to detect unapproved files. Consider using a more flexible pattern like "INFO:.*Unapproved:[ ]+[1-9]" or verify the exact output format from Apache RAT 0.17 to ensure this pattern matches correctly.
| if grep -q "^! Unapproved:" .rat-reports || grep -q "INFO: Unapproved: [1-9]" .rat-reports; then | |
| if grep -q "^! Unapproved:" .rat-reports || grep -Eq "INFO:.*Unapproved:[ ]+[1-9]" .rat-reports; then |
| if grep -q "^! Unapproved:" .rat-reports || grep -q "INFO: Unapproved: [1-9]" .rat-reports; then | ||
| echo "❌ Apache RAT check FAILED - Files with unapproved licenses found:" | ||
| echo "" | ||
| sed -n '/Files with unapproved licenses/,/^\*\*\*\*\*/p' .rat-reports | grep -v "^\*\*\*\*\*" |
There was a problem hiding this comment.
The sed command extracts content between "Files with unapproved licenses" and asterisks, but this assumes a specific output format from Apache RAT 0.17. If the output format has changed from version 0.16.1, this pattern might not match, causing the check to silently pass even when there are unapproved files. Additionally, the pattern is fragile: if RAT outputs any line with five or more asterisks before the actual section marker, the extraction will stop prematurely. Consider verifying this output format matches Apache RAT 0.17's actual output.
| sed -n '/Files with unapproved licenses/,/^\*\*\*\*\*/p' .rat-reports | grep -v "^\*\*\*\*\*" | |
| # Print detailed information about unapproved files without relying on fragile section delimiters | |
| grep "^! Unapproved:" .rat-reports || true | |
| grep "INFO: Unapproved: [1-9]" .rat-reports || true |
| java -jar apache-rat-0.17.jar -E .rat-excludes -- . > .rat-reports 2>&1 | ||
| if grep -q "^! Unapproved:" .rat-reports || grep -q "INFO: Unapproved: [1-9]" .rat-reports; then |
There was a problem hiding this comment.
The grep patterns on line 41 check for two different formats. However, if neither pattern matches (because Apache RAT 0.17 uses a different output format than expected), the condition will evaluate to false and the workflow will report success even when there are unapproved licenses. Consider adding validation to ensure the RAT command executed successfully and produced expected output before checking for failures.
|
@jomarko Can you please double-check if the below suggestions are reasonable? |
3 participants
Closes: apache/incubator-kie-issues#1670
Complete ensemble
JIRA
Referenced pull requests
Checklist
How to replicate CI configuration locally?
Build Chain tool does "simple" maven build(s), the builds are just Maven commands, but because the repositories relates and depends on each other and any change in API or class method could affect several of those repositories there is a need to use build-chain tool to handle cross repository builds and be sure that we always use latest version of the code for each repository.
build-chain tool is a build tool which can be used on command line locally or in Github Actions workflow(s), in case you need to change multiple repositories and send multiple dependent pull requests related with a change you can easily reproduce the same build by executing it on Github hosted environment or locally in your development environment. See local execution details to get more information about it.
How to retest this PR or trigger a specific build:
for pull request checks
Please add comment: Jenkins retest this
for a specific pull request check
please add comment: Jenkins (re)run [optaplanner|optaplanner-quickstarts] tests
for a full downstream build
please add comment: Jenkins run fdb
add the label
run_fdbfor a compile downstream build
please add comment: Jenkins run cdb
for a full production downstream build
please add comment: Jenkins execute product fdb
for an upstream build
please add comment: Jenkins run upstream
for quarkus branch checks
Run checks against Quarkus current used branch
Please add comment: Jenkins run quarkus-branch
for a quarkus branch specific check
Run checks against Quarkus current used branch
Please add comment: Jenkins (re)run [optaplanner|optaplanner-quickstarts] quarkus-branch
for quarkus main checks
Run checks against Quarkus main branch
Please add comment: Jenkins run quarkus-main
for a specific quarkus main check
Run checks against Quarkus main branch
Please add comment: Jenkins (re)run [optaplanner|optaplanner-quickstarts] quarkus-branch
for quarkus lts checks
Run checks against Quarkus lts branch
Please add comment: Jenkins run quarkus-lts
for a specific quarkus lts check
Run checks against Quarkus lts branch
Please add comment: Jenkins (re)run [optaplanner|optaplanner-quickstarts] quarkus-lts
for native checks
Run native checks
Please add comment: Jenkins run native
for a specific native check
Run native checks
Please add comment: Jenkins (re)run [optaplanner|optaplanner-quickstarts] native
for native lts checks
Run native checks against quarkus lts branch
Please add comment: Jenkins run native-lts
for a specific native lts check
Run native checks against quarkus lts branch
Please add comment: Jenkins (re)run [optaplanner|optaplanner-quickstarts] native-lts
CI Status
You can check OptaPlanner repositories CI status from Chain Status webpage.
How to backport a pull request to a different branch?
In order to automatically create a backporting pull request please add one or more labels having the following format
backport-<branch-name>, where<branch-name>is the name of the branch where the pull request must be backported to (e.g.,backport-7.67.xto backport the original PR to the7.67.xbranch).Once the original pull request is successfully merged, the automated action will create one backporting pull request per each label (with the previous format) that has been added.
If something goes wrong, the author will be notified and at this point a manual backporting is needed.