Add Tools section

Author: vkucera

docs/datamodel/README.md

@@ -1,5 +1,5 @@
 ---
-sort: 9
+sort: 10
 title: Appendix: data model reference
 ---
 

docs/gettingstarted/gitbasics.md

@@ -48,11 +48,7 @@ Selected sections:
 - [git - the simple guide](https://rogerdudler.github.io/git-guide/)
 - [Oh Shit, Git!?!](https://ohshitgit.com/)
 
-## Tips
-
-### Tools
-
-[Visual Studio Code (VS Code)](https://code.visualstudio.com/) – Integrated development environment
+## Tools
 
 [Meld](https://meldmerge.org/) – Visual diff and merge tool that can be used with Git commands `git difftool` and `git mergetool`
 

docs/gettingstarted/installing.md

@@ -204,6 +204,8 @@ Note that this directory tends to grow in size over time, and it is the one you
 
 When `aliBuild`is installed on your computer and your prerequisits are statisfied, you can move to the next step.
 
+See also [shell rc utilities](../tools/README.md#shell-rc-utilities).
+
 ## Prepare your source code
 
 We assume your work area is `~/alice`.
@@ -318,6 +320,8 @@ For example: `ninja PWGCF/Tasks/install`
 ninja install > ../log 2>&1 && echo "Good" || echo "Bad"
 ```
 
+See also [shell rc utilities](../tools/README.md#shell-rc-utilities).
+
 A specific executable can be built in the staging directory `stage/bin` with
 
 ```bash

docs/tools/README.md

@@ -0,0 +1,137 @@
+---
+sort: 8
+title: Tools
+---
+
+# Tools
+
+Tools which are not part of the O2(Physics) analysis framework and can make your work with it much more effective.
+
+## Setup diagnostic tool
+
+This simple Bash script prints a summary of some basic information about your system and your O2Physics installation setup.
+
+It can be useful to provide this summary when [requesting support](../troubleshooting/README.md#reporting-problems) concerning a problem with your analysis framework.
+
+Download the [`summarise_o2p_setup.sh`](summarise_o2p_setup.sh) script and run it with `bash` in your `alice` directory.
+
+## Dependency finder
+
+Tool to explore input/output dependencies between O2Physics workflows and tables.
+
+See the dedicated [Dependency finder](dependencyFinder.md) page.
+
+## O2 linter
+
+Tool to detect O2-specific (and some common C++) issues in the code.
+
+See the dedicated [O2 linter](o2linter.md) page.
+
+## Pre-commit hooks
+
+[Git hooks](https://git-scm.com/book/ms/v2/Customizing-Git-Git-Hooks) are scripts hooked to certain Git commands.
+Pre-commit hooks run when you execute `git commit` and are used to validate the staged changes that are about to be committed.
+If any hook fails, the commit is not made.
+
+In the O2Physics repository, pre-commit hooks are [configured](https://github.com/AliceO2Group/O2Physics/blob/master/.pre-commit-config.yaml) to format C++ code with [clang-format](https://clang.llvm.org/docs/ClangFormat.html) and lint it (check for issues) with [cpplint](https://github.com/cpplint/cpplint).
+This way you can make sure that your changes will pass the C++ formatting check and the cpplint check in [MegaLinter](https://megalinter.io/) when you make your pull request.
+
+### How to use pre-commit hooks
+
+1. [Install pre-commit](https://pre-commit.com/#installation) (typically `pip install pre-commit`).
+1. Enter the `O2Physics` directory.
+1. [Install the Git hook scripts](https://pre-commit.com/#3-install-the-git-hook-scripts): `pre-commit install`.
+
+Next time you execute `git commit`, the hooks will run automatically.
+
+- If the clang-format hook fails, it means your staged files have been formatted.
+- If the cpplint hook fails, the linter has found issues that need to be fixed.
+- Updated files need to be staged with `git add` before attempting `git commit` again.
+
+## [clang-tidy](https://clang.llvm.org/extra/clang-tidy/)
+
+`clang-tidy` is a clang-based C++ linter tool for diagnosing and fixing typical programming errors, like style violations or bugs.
+(See the [list of checks](https://clang.llvm.org/extra/clang-tidy/checks/list.html).)
+
+### Prerequisites for using clang-tidy
+
+- You need to have O2Physics successfully compiled.
+- Verify that there is a valid symbolic link `compile_commands.json` in the O2Physics directory pointing to the `alice/sw/BUILD/.../O2Physics` directory.
+
+### Tips
+
+#### Cleaning `#include`s
+
+The [`misc-include-cleaner`](https://clang.llvm.org/extra/clang-tidy/checks/misc/include-cleaner.html) check can fix missing and unused `#include`s.
+This helps to apply the [Include What You Use](https://github.com/AliceO2Group/O2Physics/issues/8357) principle which allows to maintain header dependencies clean.
+
+#### Sorting `#include`s
+
+The [`llvm-include-order`](https://clang.llvm.org/extra/clang-tidy/checks/llvm/include-order.html) check sorts `#include`s from most specific to least specific to ensure that the headers does not have any hidden dependencies.
+
+Unfortunately, the [LLVM include style](https://llvm.org/docs/CodingStandards.html#include-style) is incompatible with the [Google include style](https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes) applied by `cpplint`.
+
+#### Testing (and fixing) many files at once
+
+Here is an example of how to run the `misc-include-cleaner` check in parallel on all `.h` and `.cxx` files in the current directory.
+
+```bash
+parallel "clang-tidy --fix -checks=-*,misc-include-cleaner {}; echo \"{} \$?\"" ::: $(find -name "*.h" -o -name "*.cxx") > "clang-tidy.log"
+```
+
+The [`parallel`](https://www.gnu.org/software/parallel/) command is used to parallelise the execution of the `clang-tidy` command for all files.
+
+For each file, `clang-tidy` will first try to compile it and then run the enabled check(s) and fix found problems (the `--fix` option).
+The messages are redirected into `clang-tidy.log`.
+The file name and the exit code are printed below the output of `clang-tidy` so that you can get the list of files for which `clang-tidy` failed with `grep " 1$" "clang-tidy.log"`.
+
+## [Visual Studio Code (VS Code)](https://code.visualstudio.com/)
+
+Integrated development environment
+
+See [how to run O2 linter from VS Code](o2linter.md#in-visual-studio-code)
+
+### Useful extensions
+
+- [clangd](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) - C/C++ completion, navigation, and insights
+- [json](https://marketplace.visualstudio.com/items?itemName=ZainChen.json) - Json for Visual Studio Code
+- [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) - All you need to write Markdown (keyboard shortcuts, table of contents, auto preview and more)
+- [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) - Python language support with extension access points for IntelliSense (Pylance), Debugging (Python Debugger), linting, formatting, refactoring, unit tests, and more.
+- [Ruff](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) - A Visual Studio Code extension with support for the Ruff linter and formatter.
+- [ShellCheck](https://marketplace.visualstudio.com/items?itemName=timonwong.shellcheck) - Integrates ShellCheck into VS Code, a linter for Shell scripts.
+- [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) - YAML Language Support by Red Hat, with built-in Kubernetes syntax support
+- [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) - Open any folder on a remote machine using SSH and take advantage of VS Code's full feature set.
+- [PDF Viewer](https://marketplace.visualstudio.com/items?itemName=mathematic.vscode-pdf) - Portable document format (PDF) viewer for Visual Studio Code.
+
+## Shell rc utilities
+
+You can use the [`bashrc-alice.sh`](bashrc-alice.sh) file to add ALICE-related utilities in your Bash environment.
+
+The file provides:
+
+- variables needed to [configure aliBuild](../gettingstarted/installing.md#installing-alibuild)
+- utility functions for [recompiling with `ninja`](../gettingstarted/installing.md#building-partially-for-development-using-ninja)
+- utility functions for [debugging compilation and runtime failures](../troubleshooting/README.md#finding-problems)
+
+Add the [`bashrc-alice.sh`](bashrc-alice.sh) file in your home directory (`~`) and source it in your Bash environment by adding the following line in the `~/.bashrc` file.
+
+```bash
+source bashrc-alice.sh
+```
+
+## [Run 3 validation framework](https://github.com/AliceO2Group/Run3AnalysisValidation)
+
+The Run 3 validation framework is a tool for an easy execution, testing and validation of any O2Physics analysis code on large local data samples.
+
+It is extensively configurable and provides similar features as AliHyperloop, such as:
+
+- Dataset management
+- Support of linked-derived-data input
+- Automatic JSON configuration based on input properties
+- Automatic workflow topology generation based on "wagon" dependencies
+- Job parallelisation
+- Table saving
+- Output merging
+- Diagnostics
+
+Among several other utilities, it includes a [maintenance script](https://github.com/AliceO2Group/Run3AnalysisValidation?tab=readme-ov-file#keep-your-repositories-and-installations-up-to-date-and-clean) for automated maintenance of Git repositories and aliBuild packages.

docs/tools/bashrc-alice.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+
+# alice
+
+# aliBuild
+export ALIBUILD_WORK_DIR="$HOME/alice/sw"
+eval "$(alienv shell-helper)"
+
+# aliases
+alias loadali='alienv enter AliPhysics/latest'
+alias loado2='alienv enter O2/latest'
+alias loado2p='alienv enter O2Physics/latest'
+alias root='root -l'
+
+# Recompile an aliBuild development package with ninja.
+# Usage: recompile <package> <branch> <target>
+# Arguments <branch> and <target> are optional.
+# Command "recompile O2Physics" will invoke "ninja install" for the "master" branch of O2Physics.
+# Command "recompile O2Physics my-branch" will invoke "ninja install" for the "my-branch" branch of O2Physics.
+# Command "recompile O2Physics my-branch Common" will invoke "ninja Common/install" for the "my-branch" branch of O2Physics.
+recompile() {
+  # set -o xtrace # to print out each command
+  [ "$1" ] || { echo "Provide a package name"; return 1; }
+  package="$1"
+  branch="master"
+  [ "$2" ] && branch="$2"
+  target=""
+  target_name="all"
+  [ "$3" ] && { target="$3/"; target_name="$3"; }
+  dir_pwd=$(pwd)
+  dir_build="$ALIBUILD_WORK_DIR/BUILD/${package}-latest-${branch}/${package}"
+  log="$(dirname "$dir_build")/log"
+  log_err="$(dirname "$dir_build")/log_err"
+  cd "$dir_build" || { echo "Could not enter $dir_build"; return 1; }
+  direnv allow || { echo "Failed to allow direnv"; return 1; }
+  eval "$(direnv export "$SHELL")"
+  echo "Recompiling ${package}_${branch}_${target_name}..."
+  start=$(date +%s)
+  ninja "${target}install" > "$log" 2>&1
+  ec=$?
+  end=$(date +%s)
+  echo "Compilation exited with: $ec"
+  echo "See the log at: $log"
+  if [ "$ec" != "0"  ]; then
+    grep -e "FAILED:" -e "error:" "$log" > "$log_err"
+    echo "See the errors at: $log_err"
+  fi
+  echo "Took $((end - start)) seconds."
+  cd "$dir_pwd" || return 1
+  # set +o xtrace
+  return $ec
+}
+
+# Recompile O2 with ninja.
+recompile-o2() { recompile "O2" "$1" "$2"; }
+# Recompile O2Physics with ninja.
+recompile-o2p() { recompile "O2Physics" "$1" "$2"; }
+
+# Find the workflow that produces a given table.
+# Limited functionality. Use find_dependencies.py for a full search.
+find-o2-table-producer() {
+  # Check that we are inside the O2 or the O2Physics directory.
+  [[ "$PWD/" != *"/O2"*"/"* ]] && { echo "You must be inside the O2 or the O2Physics directory."; return 1; }
+  [ ! "$1" ] && { echo "Provide a table name."; return 1; }
+  # Find files that produce the table.
+  table="$1"
+  echo "Table $table is produced in:"
+  files=$(grep -r -i --include="*.cxx" "<aod::$table>" | grep -E 'Produces|Spawns' | cut -d: -f1 | sort -u)
+  for f in $files; do
+    # Extract the workflow name from the CMakeLists.txt in the same directory.
+    wf=$(grep -B 1 "$(basename "$f")" "$(dirname "$f")/CMakeLists.txt" | head -n 1 | cut -d\( -f2)
+    echo "$wf in $f"
+  done
+}
+
+# Find compilation error messages in a compilation log.
+debug-o2-compile() {
+  [ "$1" ] || { echo "Provide a log file"; return 1; }
+  grep -n -e "FAILED:" -e "error:" -e "warning:" "$1"
+}
+
+# Find runtime error messages in an execution log.
+debug-o2-run() {
+  [ "$1" ] || { echo "Provide a log file"; return 1; }
+  grep -n -e "\\[ERROR\\]" -e "\\[FATAL\\]" -e "segmentation" -e "Segmentation" -e "SEGMENTATION" -e "command not found" -e "Program crashed" -e "Error:" -e "Error in " -e "\\[WARN\\]" -e "Warning in " "$1"
+}