Merge pull request #7300 from itemko/artemry/mellanox-ci-with-azure-pipelines-review

Reworked Mellanox Open MPI CI with Azure Pipelines

Author: jsquyres

.ci/README.md

@@ -0,0 +1,18 @@
+# Open MPI Continuous Integration (CI) Services
+## Mellanox Open MPI CI
+[![Build Status](https://dev.azure.com/mlnx-swx/mellanox-ompi-ci-project/_apis/build/status/mellanox-ompi-ci-pipeline?branchName=master)](https://dev.azure.com/mlnx-swx/mellanox-ompi-ci-project/_build/latest?definitionId=6&branchName=master)
+### Scope
+[Mellanox](https://www.mellanox.com/) Open MPI CI is intended to verify Open MPI with recent Mellanox SW components ([Mellanox OFED](https://www.mellanox.com/page/products_dyn?product_family=26), [UCX](https://www.mellanox.com/page/products_dyn?product_family=281&mtag=ucx) and other [HPC-X](https://www.mellanox.com/page/products_dyn?product_family=189&mtag=hpc-x) components) in the Mellanox lab environment.
+
+CI is managed by [Azure Pipelines](https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops) service.
+
+Mellanox Open MPI CI includes:
+* Open MPI building with internal stable engineering versions of UCX and HCOLL. The building is run in Docker-based environment.
+* Sanity functional testing.
+### How to Run CI
+Mellanox Open MPI CI is triggered upon the following events:
+* Push a commit into the master branch. CI status and log files are available on the Azure DevOps server.
+* Create a pull request (PR). CI status is visible in the PR status. CI is restarted automatically upon each new commit within the PR. CI status and log files are also available on the Azure DevOps server.
+* Trigger CI with special PR comments (for example, `/azp run`). Comment triggers are available only if the comment author has write permission to the PR target repo. Detailed information about comment triggers is available in the official Azure DevOps [documentation](https://docs.microsoft.com/en-us/azure/devops/pipelines/repos/github?view=azure-devops&tabs=yaml#comment-triggers).
+### Support
+In case of any issues, questions or suggestions please contact to [Mellanox Open MPI CI support team](mailto:[email protected];[email protected]).

.ci/mellanox/azure-pipelines.yml

@@ -0,0 +1,51 @@
+# Per-commit
+trigger:
+  - master
+# All PRs
+pr:
+  branches:
+    include:
+    - '*'
+
+pool:
+  name: Default
+  demands: AGENT_CI_CAPABILITY -equals ompi
+
+variables:
+  # TODO: change to the main repo and master branch
+  #ompi_jenkins_scripts_git_repo_url: https://github.com/mellanox-hpc/jenkins_scripts.git
+  ompi_jenkins_scripts_git_repo_url: https://github.com/itemko/jenkins_scripts.git
+  #ompi_jenkins_scripts_git_branch: master
+  ompi_jenkins_scripts_git_branch: artemry/ompi_review
+  # Enable debug information, supported values: true, false
+  debug: true
+
+jobs:
+- job: mellanox_ompi_ci
+  displayName: Mellanox Open MPI CI
+  timeoutInMinutes: 240
+  container:
+    image: rdmz-harbor.rdmz.labs.mlnx/hpcx/ompi_ci:latest
+    options: -v /hpc/local:/hpc/local -v /opt:/opt --uts=host --ipc=host --ulimit stack=67108864
+      --ulimit memlock=-1 --security-opt seccomp=unconfined --cap-add=SYS_ADMIN --device=/dev/infiniband/
+  steps:
+  - task: DeleteFiles@1
+    displayName: Cleanup workspace folder
+    inputs:
+      sourceFolder: $(Pipeline.Workspace)
+      contents: |
+        **/*
+  - checkout: self
+    submodules: true
+    path: ompi
+  - bash: |
+      set -eE
+      [ "$(debug)" = "true" ] && set -x
+      cd $(Pipeline.Workspace)
+      git clone $(ompi_jenkins_scripts_git_repo_url)
+      cd $(Pipeline.Workspace)/jenkins_scripts && git checkout $(ompi_jenkins_scripts_git_branch)
+      export WORKSPACE=$(Pipeline.Workspace)/ompi
+      # TODO: rework ompi_test.sh to avoid Jenkins mentions
+      export JENKINS_RUN_TESTS=yes
+      $(Pipeline.Workspace)/jenkins_scripts/jenkins/ompi/ompi_test.sh
+    displayName: Build and test Open MPI

.gitignore

@@ -6,6 +6,7 @@
 .cdt*
 .project
 .gdb*
+.idea
 
 .hgrc
 .hgignore