Merge pull request #107 from Turiok/integration_test

Add an integration test

Author: louisjdmartin

.gitignore

@@ -7,6 +7,7 @@ release.properties
 dependency-reduced-pom.xml
 buildNumber.properties
 .mvn/timing.properties
+*.log
 
 # Avoid ignoring Maven wrapper jar file (.jar files are usually ignored)
 !/.mvn/wrapper/maven-wrapper.jar
@@ -29,4 +30,4 @@ buildNumber.properties
 # Ignore IntelliJ files
 .idea
 *.iml
-.vscode
+.vscode

CONTRIBUTING.md

@@ -1,25 +1,30 @@
 # Contributing to i-Code CNES plugin for SonarQube
 First off, thanks for taking the time to contribute!
-The following is a set of guidelines for contributing to i-Code CNES plugin for SonarQube, which are hosted in the [Lequal Organization](https://github.com/lequal) on GitHub. These are mostly guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request.
+The following is a set of guidelines for contributing to i-Code CNES plugin for SonarQube, which are hosted in the [CATLab organization](https://github.com/cnescatlab) on GitHub. These are mostly guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request.
 
 #### Table Of Contents
-+ [Code of Conduct](#code-of-conduct)
-+ [How Can I Contribute?](#how-can-i-contribute)  
-  * [Reporting Bugs](#reporting-bugs)  
-  * [Suggesting Enhancements](#suggesting-enhancements)  
-  * [Pull Requests](#pull-requests)
-  * [Git Commit Messages](#git-commit-messages)
+- [Contributing to i-Code CNES plugin for SonarQube](#contributing-to-i-code-cnes-plugin-for-sonarqube)
+      - [Table Of Contents](#table-of-contents)
+  - [Code of Conduct](#code-of-conduct)
+  - [How Can I Contribute?](#how-can-i-contribute)
+    - [Reporting Bugs](#reporting-bugs)
+      - [How Do I Submit A (Good) Bug Report?](#how-do-i-submit-a-good-bug-report)
+    - [Suggesting Enhancements](#suggesting-enhancements)
+      - [How Do I Submit A (Good) Enhancement Suggestion?](#how-do-i-submit-a-good-enhancement-suggestion)
+    - [Pull Requests](#pull-requests)
+    - [Git Commit Messages](#git-commit-messages)
+  - [Management of contributions](#management-of-contributions)
 
 ## Code of Conduct
-This project and everyone participating in it is governed by the [Lequal Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code.
+This project and everyone participating in it is governed by the [CATLab Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to us or [GitHub](https://github.com/contact/report-content).
 
 ## How Can I Contribute?
 
 ### Reporting Bugs
-Before creating bug reports, please check if the problem has already been reported. 
+Before creating bug reports, please check if the problem has already been reported.
 
 #### How Do I Submit A (Good) Bug Report?
-Bugs are tracked as [GitHub issues](https://guides.github.com/features/issues/). 
+Bugs are tracked as [GitHub issues](https://guides.github.com/features/issues/).
 Provide information by filling [the template](.github/ISSUE_TEMPLATE/bug_report.md).
 
 Explain the problem and include additional details to help maintainers reproduce the problem:
@@ -32,7 +37,7 @@ Explain the problem and include additional details to help maintainers reproduce
 
 Provide more context by answering these questions:
 * **Did the problem start happening recently** (e.g. after updating to a new version of the plugin or SonarQube) or was this always a problem?
-* If the problem started happening recently, **can you reproduce the problem in an older version?** What's the most recent version in which the problem doesn't happen? You can download older versions from [the releases page](https://github.com/lequal/sonar-icode-cnes-plugin/releases).
+* If the problem started happening recently, **can you reproduce the problem in an older version?** What's the most recent version in which the problem doesn't happen? You can download older versions from [the releases page](https://github.com/cnescatlab/sonar-icode-cnes-plugin/releases).
 * **Can you reliably reproduce the issue?** If not, provide details about how often the problem happens and under which conditions it normally happens.
 
 Include details about your configuration and environment:
@@ -72,8 +77,69 @@ Enhancement suggestions are tracked as [GitHub issues](https://guides.github.com
 
 All contributions are welcome. They are made via a *pull request* on the branch `dev` which is the branch of the next version.
 
-* pull request with **major** changes must be approved by at least one maintainer of each team and the LEQUAL.
+* pull request with **major** changes must be approved by at least one maintainer of each team and the CATLab.
 
 * pull requests with **minor** changes must be approved by at least one organization's member.
- 
+
 All maintainers have the ability to merge *pull requests* on the `dev` branch. If several maintainers belong to the same team, their validation only counts for one organization.
+
+# Development
+## How to build
+
+This plugin is mainly developped in Java* and build with maven**. It's main dependencies are [fr.cnes.icode.icode-library](https://github.com/cnescatlab/i-CodeCNES) and libraries for [SonarQube](https://docs.sonarsource.com/sonarqube-community-build/extension-guide/developing-a-plugin/plugin-basics/)
+
+> \* Depending the version, the JDK version is written in the [pom.xml](pom.xml) ans the SonarQube compatibility is in the matrix: [matrix](README.md#run-i-code-automatically)
+>
+> \*\* the build for the release version are validated in maven 3.9.9
+
+
+The icode-library is available on github repository. You'll have to configured your maven settings.xml with a new repository like this:
+
+```xml
+<repositories>
+  <repository>
+    <id>github</id>
+    <url>https://maven.pkg.github.com/cnescatlab/*</url>
+  </repository>
+</repositories>
+```
+
+For all other dependencies, maven [central](https://central.sonatype.com/) is enough.
+
+After that, you only have to launch the build command:
+
+```bash
+mvn package
+```
+
+The jar will be in the **target** folder.
+
+## How to launch integration test
+
+An integration test is written to validate the plugin in a SonarQube dockerized instance and with a Fortran scan.
+
+** [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) is needed for this integration test
+
+There is 2 scripts to make the test:
+- The [it.sh](it/it.sh) to launch a SonarQube docker instance and install the local sonar-icode-cnes-plugin-<MAVEN_VERSION>.jar in this instance.
+- the [audit.sh](it/audit.sh) to make a sonar-scanner launch a Fortran 77 and Fortran 90 code in the [src](it/src/) folder. This script check f77 and f90 issues a shown by SonarQube using the API.
+
+
+The it script can use parameters:
+```bash
+$> ./it.sh -h
+
+Usage: ./it.sh [sSh]
+
+-h : Display help
+-s [SONAR-SCANNER] : Take Sonar-scanner tag image from https://hub.docker.com/r/sonarsource/sonar-scanner-cli. Default is the latest tag
+-S [SONARQUBE] : Take SonarQube tag image from https://hub.docker.com/_/sonarqube. Default is the community tag
+```
+
+Example of launching :
+```bash
+cd it
+./it.sh -s 11.4.0.2044_7.2.0 -S lts-community
+```
+
+> Be careful to choose version depending the [compatibility matrix](README.md#run-i-code-automatically)

README.md

@@ -81,7 +81,7 @@ Please report issues at https://github.com/leqcnescatlabual/i-CodeCNES/issues
 ````
 
 ### How to contribute
-If you experienced a problem with the plugin please open an issue. Inside this issue please explain us how to reproduce this issue and paste the log. 
+If you experienced a problem with the plugin please open an issue. Inside this issue please explain us how to reproduce this issue and paste the log.
 
 If you want to do a PR, please put inside of it the reason of this pull request. If this pull request fix an issue please insert the number of the issue or explain inside of the PR how to reproduce this issue.
 

it/Dockerfile-auditor

@@ -0,0 +1,36 @@
+ARG SCANNER_VERSION=latest
+FROM sonarsource/sonar-scanner-cli:${SCANNER_VERSION} AS builder
+
+FROM eclipse-temurin:21-jre-alpine
+
+ARG SONAR_SCANNER_HOME=/opt/sonar-scanner
+ENV HOME=/tmp \
+    SONAR_SCANNER_HOME=${SONAR_SCANNER_HOME} \
+    XDG_CONFIG_HOME=/tmp \
+    SONAR_USER_HOME=${SONAR_SCANNER_HOME}/.sonar \
+    PATH=${SONAR_SCANNER_HOME}/bin:${PATH} \
+    SRC_PATH=/usr/src \
+    LANG=C.UTF-8 \
+    LC_ALL=C.UTF-8 \
+    PYTHONUNBUFFERED=1
+
+WORKDIR /usr/src/myapp/it
+
+USER root
+# Copy Scanner installation from builder image
+COPY --from=builder /opt/sonar-scanner /opt/sonar-scanner
+
+
+RUN apk update --no-cache && \
+    apk add --update --no-cache -q curl gcc jq libffi-dev musl-dev openssl-dev python3 py3-requests shellcheck
+
+RUN set -eux && \
+    addgroup --gid 1000 scanner-cli && \
+    adduser --uid 1000 --ingroup scanner-cli --disabled-password --no-create-home --gecos "" scanner-cli && \
+    mkdir -p "${SRC_PATH}" "${SONAR_USER_HOME}" "${SONAR_USER_HOME}/cache" && \
+    chown -R scanner-cli:scanner-cli "${SONAR_SCANNER_HOME}" "${SRC_PATH}" && \
+    chmod -R 555 "${SONAR_SCANNER_HOME}" && \
+    chmod -R 754 "${SRC_PATH}" "${SONAR_USER_HOME}"
+
+USER scanner-cli
+COPY --chown=scanner-cli:scanner-cli it /usr/src/myapp/it

it/audit.sh

@@ -0,0 +1,90 @@
+#!/bin/sh -e
+
+# ----------------------------------
+# Colors
+# ----------------------------------
+NOCOLOR='\033[0m'
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+ORANGE='\033[0;33m'
+
+# Function to get value from a property file
+# arg 1 = the property
+# arg 2 = the file path
+function prop {
+    grep "${1}" ${2} | cut -d'=' -f2
+}
+
+# Configure sonar-scanner
+export SONAR_HOST_URL="http://sonarqube:9000"
+export SONAR_ADMIN_LOGIN="admin"
+export SONAR_ADMIN_PWD="admin"
+
+# Generate Analysis token
+echo "Generating analysis token..."
+# Use an UUID for token name. It's useful to launch the audit several time on the same SonarQube execution
+uuid=$(cat /proc/sys/kernel/random/uuid)
+export SONAR_TOKEN=$(curl -su "$SONAR_ADMIN_LOGIN:$SONAR_ADMIN_PWD" -XPOST "$SONAR_HOST_URL/api/user_tokens/generate?name=$uuid&type=GLOBAL_ANALYSIS_TOKEN" | jq -r '.token')
+echo $SONAR_TOKEN
+# Audit code
+echo "Launching scanner..."
+cd /usr/src/myapp/it
+sonar-scanner -X -Dsonar.qualitygate.wait 2>&1 | tee /tmp/scanner.log
+
+if [ $? -ne 0 ]
+then
+    echo "${RED}Error scanning Shell scripts${NOCOLOR}" >&2
+    exit 1
+fi
+
+# Check for warnings
+if grep -q "^WARN: " /tmp/scanner.log
+then
+    echo -e "${ORANGE}Warnings found ${NOCOLOR}" >&2
+    exit 1
+fi
+
+# Sleep a little because SonarQube needs some time to ingest the audit results
+sleep 10
+
+export SONAR_PROJECT_KEY=$(prop 'sonar.projectKey' sonar-project.properties)
+echo "SONAR_PROJECT_KEY: $SONAR_PROJECT_KEY"
+
+# Check audit result
+echo "Checking result..."
+python3 << EOF
+from __future__ import print_function
+import requests
+import sys
+
+r = requests.get('http://sonarqube:9000/api/issues/search?componentKeys=$SONAR_PROJECT_KEY:src/clanhb.f&statuses=OPEN', auth=('$SONAR_ADMIN_LOGIN', '$SONAR_ADMIN_PWD'))
+if r.status_code != 200:
+    print('Invalid server response: ' + str(r.status_code), file=sys.stderr)
+    sys.exit(1)
+
+data = r.json()
+
+if data['total'] != 100:
+    print('Wrong total number of issues: ' + str(data['total']), file=sys.stderr)
+    sys.exit(1)
+
+issues = 0
+if 'f77-rules' in data['issues'][0]['rule'] and data['issues'][0]['line'] == 1:
+    issues += 1
+
+r = requests.get('http://sonarqube:9000/api/issues/search?componentKeys=$SONAR_PROJECT_KEY:src/clanhb.f90&statuses=OPEN', auth=('$SONAR_ADMIN_LOGIN', '$SONAR_ADMIN_PWD'))
+if r.status_code != 200:
+    print('Invalid server response: ' + str(r.status_code), file=sys.stderr)
+    sys.exit(1)
+
+data = r.json()
+
+if data['total'] != 197:
+    print('Wrong total number of issues: ' + str(data['total']), file=sys.stderr)
+    sys.exit(1)
+if 'f90-rules' in data['issues'][0]['rule'] and data['issues'][0]['line'] == 1:
+    issues += 1
+
+
+sys.exit(0 if issues == 2 else 1)
+EOF

it/docker-compose.yml

@@ -0,0 +1,23 @@
+---
+services:
+  sonarqube:
+    image: sonarqube:${SONARQUBE_VERSION:-community}
+    ports:
+      - "9000:9000"
+    environment:
+      ES_JAVA_OPTS: "-Xms750m -Xmx750m"
+      SONARQUBE_VERSION: ${SONARQUBE_VERSION:-community}
+    security_opt:
+      - seccomp:unconfined
+  auditor:
+    image: auditor:${SCANNER_VERSION:-latest}
+    build:
+      context: ..
+      dockerfile: it/Dockerfile-auditor
+      args:
+        SCANNER_VERSION: ${SCANNER_VERSION:-latest}
+    links:
+      - sonarqube
+    command: /bin/sh -e /usr/src/myapp/it/audit.sh
+    environment:
+      SCANNER_VERSION: ${SCANNER_VERSION:-latest}

it/it.sh

@@ -0,0 +1,93 @@
+#!/bin/bash
+
+usage(){
+    echo -e "\nUsage: $0 [sSh] \n"
+    echo "-h : Display help"
+    echo "-s [SONAR-SCANNER] : Take Sonar-scanner tag image from https://hub.docker.com/r/sonarsource/sonar-scanner-cli". Default is the latest tag
+    echo "-S [SONARQUBE] : Take SonarQube tag image from https://hub.docker.com/_/sonarqube". Default is the community tag
+}
+
+OPTSTRING=":s:S:h"
+
+while getopts ${OPTSTRING} opt; do
+  case ${opt} in
+    s)
+      export SCANNER_VERSION=$OPTARG
+      ;;
+    S)
+      export SONARQUBE_VERSION=$OPTARG
+      ;;
+    h)
+      usage
+      exit 0
+      ;;
+    :)
+      echo "Option -${OPTARG} requires an argument."
+      usage
+      exit 1
+      ;;
+    ?)
+      echo "Invalid option: -${OPTARG}."
+      usage
+      exit 1
+      ;;
+  esac
+done
+
+
+export SCRIPT_DIR=`dirname $0`
+
+# Clean-up if needed
+echo "Cleanup..."
+docker-compose -f $SCRIPT_DIR/docker-compose.yml down
+
+# Start containers
+echo "Starting SonarQube..."
+docker-compose -f $SCRIPT_DIR/docker-compose.yml up -d sonarqube
+CONTAINER_NAME=$(docker ps --format "{{.Names}}" | grep 'it-sonarqube-1.*' | head -1)
+# Wait for SonarQube to be up
+grep -q "SonarQube is operational" <(docker logs --follow --tail 0 $CONTAINER_NAME)
+echo "SonarQube started!"
+
+# Copy the plugins
+MAVEN_VERSION=$(grep '<version>' $SCRIPT_DIR/../pom.xml | head -1 | sed 's/<\/\?version>//g'| awk '{print $1}')
+echo "Installing the plugin Icode version $MAVEN_VERSION"
+docker cp $SCRIPT_DIR/../target/sonar-icode-cnes-plugin-$MAVEN_VERSION.jar $CONTAINER_NAME:/opt/sonarqube/extensions/plugins
+# Restart SonarQube
+docker-compose -f $SCRIPT_DIR/docker-compose.yml restart sonarqube
+# Wait for SonarQube to be up
+grep -q "SonarQube is operational" <(docker logs --follow --tail 0 $CONTAINER_NAME)
+# Check plug-in installation
+docker exec -u root $CONTAINER_NAME bash -c "if grep -q Alpine /etc/issue; then apk update && apk add -q curl; fi"
+docker exec -u root $CONTAINER_NAME bash -c "if grep -q Ubuntu /etc/issue; then apt-get update && apt-get install -y curl; fi"
+if ! docker exec $CONTAINER_NAME curl -su admin:admin http://localhost:9000/api/plugins/installed | python -c '
+import sys
+import json
+plugins_count = 0
+plugins_expected = ["icode"]
+data = json.loads(sys.stdin.read())
+if "plugins" in data:
+    for plugin in data["plugins"]:
+      if plugin["key"] in plugins_expected:
+        plugins_count += 1
+if plugins_count == len(plugins_expected):
+  sys.exit(0)
+else:
+  sys.exit(1)
+'
+then
+    echo "Plugin not installed" >&2
+    exit 1
+fi
+echo "Plugin successfully installed!"
+
+# Audit code
+echo "Audit test scripts..."
+docker-compose -f $SCRIPT_DIR/docker-compose.yml up --build --exit-code-from auditor auditor
+AUDIT_STATUS=$?
+
+# Delete containers
+echo "Cleanup..."
+docker-compose -f $SCRIPT_DIR/docker-compose.yml down
+
+exit $AUDIT_STATUS

it/sonar-project.properties

@@ -0,0 +1,5 @@
+sonar.projectKey=icode
+sonar.projectName=I-Code
+sonar.projectVersion=1.0
+sonar.sources=src
+sonar.scm.disabled=True