Skip to content

Commit 6f301c5

Browse files
authored
Merge pull request #30 from jpartlow/exp-gha-acceptance
Exp gha acceptance
2 parents 62c7e3c + 6c0999f commit 6f301c5

File tree

8 files changed

+243
-48
lines changed

8 files changed

+243
-48
lines changed

.github/workflows/acceptance.yaml

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
name: Beaker Acceptance Tests
2+
run-name: |-
3+
Beaker acceptance tests for ${{ inputs.pre-release-build && 'pre-release' || 'release' }} package
4+
${{ (inputs.pre-release-build && inputs.version) ||
5+
format(' collection: "{0}", version: "{1}" ',
6+
inputs.collection,
7+
((inputs.version == '') && 'latest') || inputs.version) }}
8+
9+
on:
10+
workflow_dispatch:
11+
inputs:
12+
pre-release-build:
13+
description: |-
14+
(Pre-release Build) Whether to test unreleased version
15+
packages from the artifacts server, or released packages
16+
from the given collection.
17+
18+
If this is true, version must be a valid version, not
19+
latest, and collection is ignored. The workflow will
20+
download and install the matching openvox-agent package file
21+
from the artifacts-url server.
22+
23+
If this is false, version and collection must match, and the
24+
workflow will install the given openvox collection package
25+
and then let the system package manager install the latest
26+
or version package from the collection repository.
27+
default: true
28+
type: boolean
29+
version:
30+
description: '(Version) Package version to test. (required if Pre-release Build is true)'
31+
type: string
32+
collection:
33+
description: '(Collection) OpenVox collection to use. (ignored if Pre-release Build is true)'
34+
default: 'openvox8'
35+
type: string
36+
artifacts-url:
37+
description: 'URL to the artifacts server. (used if Pre-release Build is true)'
38+
default: 'https://s3.osuosl.org/openvox-artifacts'
39+
type: string
40+
41+
permissions:
42+
contents: read
43+
44+
env:
45+
RUBY_VERSION: '3.3'
46+
47+
jobs:
48+
acceptance:
49+
name: Acceptance Tests
50+
runs-on: ubuntu-latest
51+
strategy:
52+
matrix:
53+
os:
54+
- [ubuntu, '18.04']
55+
- [ubuntu, '20.04']
56+
- [ubuntu, '22.04']
57+
- [ubuntu, '24.04']
58+
- [debian, '10']
59+
- [debian, '11']
60+
- [debian, '12']
61+
steps:
62+
- uses: actions/checkout@v4
63+
with:
64+
fetch-depth: 0
65+
- id: vm-cluster
66+
uses: jpartlow/kvm_automation_tooling@main
67+
with:
68+
os: ${{ matrix.os[0] }}
69+
os-version: ${{ matrix.os[1] }}
70+
os-arch: x86_64
71+
host-root-access: true
72+
ruby-version: ${{ env.RUBY_VERSION }}
73+
install-openvox: true
74+
openvox-version: ${{ github.event.inputs.version }}
75+
openvox-collection: ${{ github.event.inputs.collection }}
76+
openvox-released: ${{ github.event.inputs.pre-release-build == 'false' }}
77+
openvox-artifacts-url: ${{ github.event.inputs.artifacts-url }}
78+
vms: |-
79+
[
80+
{
81+
"role": "agent",
82+
"count": 1,
83+
"cpus": 2,
84+
"mem_mb": 4096
85+
}
86+
]
87+
- name: Install Ruby and Run Bundler
88+
uses: ruby/setup-ruby@v1
89+
with:
90+
ruby-version: ${{ env.RUBY_VERSION }}
91+
working-directory: acceptance
92+
bundler-cache: true
93+
- name: Construct hosts.yaml
94+
working-directory: kvm_automation_tooling
95+
env:
96+
HOSTS_YAML: ${{ github.workspace }}/acceptance/hosts.yaml
97+
run: |-
98+
bundle exec bolt plan run \
99+
kvm_automation_tooling::dev::generate_beaker_hosts_file \
100+
--inventory terraform/instances/inventory.*.yaml \
101+
hosts_yaml="${HOSTS_YAML}"
102+
- name: Run Beaker
103+
working-directory: acceptance
104+
run: |-
105+
# TODO The ssh keyfile here is from the kvm_automation_tooling
106+
# action...this should be cleaned up to be more discoverable
107+
# or supplied as an input perhaps.
108+
bundle exec beaker init --hosts hosts.yaml \
109+
--preserve-hosts always --keyfile ~/.ssh/ssh-id-test \
110+
--pre-suite pre-suite/configure_type_defaults.rb \
111+
--tests tests
112+
# The provision step is still needed here, see notes in the
113+
# kvm_automation_tooling/templates/beaker-hosts.yaml.epp
114+
# template...
115+
bundle exec beaker provision
116+
bundle exec beaker exec

.github/workflows/checks.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,4 @@ jobs:
2929
run: |
3030
gem update --system --silent --no-document
3131
bundle install --jobs 4 --retry 3
32-
- run: bundle exec rake ${{ matrix.check }}
32+
- run: bundle exec rake ${{ matrix.check }} --trace

README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ The Puppet Agent
77
* License
88
* Code Owners
99
* Running Tests
10+
* Github Actions
1011

1112
Overview
1213
---
@@ -144,3 +145,13 @@ See [CODEOWNERS](CODEOWNERS)
144145
Running Tests
145146
---
146147
See [Acceptance README](acceptance/README.md)
148+
149+
Github Actions
150+
---
151+
152+
PR validation runs a pair of GHA jobs that check commit messages and
153+
run Rubocop (.github/workflows/checks.yml) via the Rakefile.
154+
155+
There is also a manual acceptance GHA pipeline that runs the Beaker
156+
acceptance suite:
157+
See [GHA_ACCEPTANCE](acceptance/GHA_ACCEPTANCE.md)

Rakefile

Lines changed: 60 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,24 @@
11
require 'open3'
22

3-
RED = "\033[31m"
4-
GREEN = "\033[32m"
5-
RESET = "\033[0m"
3+
RED = "\033[31m".freeze
4+
GREEN = "\033[32m".freeze
5+
RESET = "\033[0m".freeze
66

7+
# Let's ignore rubocop here until such a time as we either muzzle it
8+
# or get these helpers more centralized and tested.
9+
# rubocop:disable Metrics/AbcSize,Metrics/CyclomaticComplexity,Metrics/MethodLength
710
def run_command(cmd, silent: true, print_command: false, report_status: false)
8-
puts "#{GREEN}Running #{cmd}#{RESET}" if print_command
11+
cmd_string = cmd.is_a?(String) ? cmd : cmd.join(' ')
12+
puts "#{GREEN}Running #{cmd_string}#{RESET}" if print_command
913
output = ''
10-
Open3.popen2e(cmd) do |_stdin, stdout_stderr, thread|
14+
Open3.popen2e(*Array(cmd)) do |_stdin, stdout_stderr, thread|
1115
stdout_stderr.each do |line|
1216
puts line unless silent
1317
output += line
1418
end
1519
exitcode = thread.value.exitstatus
1620
unless exitcode.zero?
17-
err = "#{RED}Command failed! Command: #{cmd}, Exit code: #{exitcode}"
21+
err = "#{RED}Command failed! Command: #{cmd_string}, Exit code: #{exitcode}"
1822
# Print details if we were running silent
1923
err += "\nOutput:\n#{output}" if silent
2024
err += RESET
@@ -24,6 +28,7 @@ def run_command(cmd, silent: true, print_command: false, report_status: false)
2428
end
2529
output.chomp
2630
end
31+
# rubocop:enable Metrics/AbcSize,Metrics/CyclomaticComplexity,Metrics/MethodLength
2732

2833
Dir.glob(File.join('tasks/**/*.rake')).each { |file| load file }
2934

@@ -44,45 +49,58 @@ end
4449

4550
desc "verify that commit messages match CONTRIBUTING.md requirements"
4651
task(:commits) do
47-
# This rake task looks at the summary from every commit from this branch not
48-
# in the branch targeted for a PR. This is accomplished by using the
49-
# TRAVIS_COMMIT_RANGE environment variable, which is present in travis CI and
50-
# populated with the range of commits the PR contains. If not available, this
51-
# falls back to `main..HEAD` as a next best bet as `main` is unlikely to
52-
# ever be absent.
53-
#
54-
# When we move to GH actions, use `GITHUB_BASE_REF` to resolve the merge base
52+
# Use `GITHUB_BASE_REF` to resolve the merge base
5553
# ref, which is the common ancestor between the base branch and PR. Then do
56-
# git log for all of the commits in `HEAD` that are not in the base ref
57-
#
58-
# baseref = %x{git merge-base HEAD $GITHUB_BASE_REF}
59-
# commits = "#{baseref}..HEAD"
60-
commits = ENV['TRAVIS_COMMIT_RANGE'].nil? ? 'main..HEAD' : ENV['TRAVIS_COMMIT_RANGE'].sub(/\.\.\./, '..')
61-
%x{git log --no-merges --pretty=%s #{commits}}.each_line do |commit_summary|
62-
error_message=<<-HEREDOC
63-
\n\n\n\tThis commit summary didn't match CONTRIBUTING.md guidelines:\n \
64-
\n\t\t#{commit_summary}\n \
65-
\tThe commit summary (i.e. the first line of the commit message) should start with one of:\n \
66-
\t\t(docs)\n \
67-
\t\t(maint)\n \
68-
\t\t(packaging)\n \
69-
\t\t(<ANY PUBLIC JIRA TICKET>)\n \
70-
\n\tThis test for the commit summary is case-insensitive.\n\n\n
54+
# git log for all of the commits in `HEAD` that are not in the base ref.
55+
github_base_ref = ENV.fetch('GITHUB_BASE_REF', 'main')
56+
github_base_ref = 'main' if github_base_ref.empty?
57+
# There's still something odd here. After action/checkout, 'main' by itself raises
58+
# objections from git like: `fatal: Not a valid object name main`.
59+
# but 'origin/main' works. However, if github_base_ref were a sha,
60+
# origin/SHA would fail as a reference...
61+
git_merge_base_cmd = ['git', 'merge-base', 'HEAD', "origin/#{github_base_ref}"]
62+
baseref = run_command(git_merge_base_cmd)
63+
commits = "#{baseref}..HEAD"
64+
git_log_cmd = ['git', 'log', '--no-merges', '--pretty=%s', commits]
65+
commit_lines = run_command(git_log_cmd, silent: false, print_command: true)
66+
67+
commit_lines.each_line do |commit_summary|
68+
error_message = <<~HEREDOC
69+
\n\n\n\tThis commit summary didn't match CONTRIBUTING.md guidelines:
70+
\n\t\t#{commit_summary}
71+
\tThe commit summary (i.e. the first line of the commit message) should start with one of:
72+
\t\t(docs)
73+
\t\t(maint)
74+
\t\t(packaging)
75+
\t\t(<gh-#>) (An existing github ticket ref)
76+
\n\tThis test for the commit summary is case-insensitive.\n
7177
HEREDOC
7278

73-
if /^\((maint|doc|docs|packaging|pa-\d+)\)|revert|bumping|merge|promoting/i.match(commit_summary).nil?
74-
ticket = commit_summary.match(/^\(([[:alpha:]]+-[[:digit:]]+)\).*/)
75-
if ticket.nil?
76-
raise error_message
77-
else
78-
require 'net/http'
79-
require 'uri'
80-
uri = URI.parse("https://tickets.puppetlabs.com/browse/#{ticket[1]}")
81-
response = Net::HTTP.get_response(uri)
82-
if response.code != "200"
83-
raise error_message
84-
end
85-
end
79+
commit_format_regex = /
80+
^\((?:maint|doc|docs|packaging|(([[:alpha:]]+)-(\d+)))\)|
81+
revert|bumping|merge|promoting
82+
/xi
83+
match = commit_summary.match(commit_format_regex)
84+
raise error_message if match.nil?
85+
86+
ticket = match[1]
87+
next if ticket.nil?
88+
project = match[2]
89+
# Could be an old PUP- or other Jira, and while there is still a
90+
# https://puppet.atlassian.net archive, determining that a
91+
# particular ticket does not exist is more involved that getting a
92+
# 200 on a call like https://puppet.atlassian.net/browse/PUP-1234
93+
next if project != 'gh'
94+
ticket_number = match[3]
95+
96+
require 'net/http'
97+
require 'uri'
98+
uri = URI.parse("https://github.com/openvoxproject/openvox-agent/issues/#{ticket_number}")
99+
response = Net::HTTP.get_response(uri)
100+
if response.code != "200"
101+
ticket_error = "\t#{RED}Did not find a ticket matching #{ticket} at #{uri}#{RESET}\n\n"
102+
raise error_message + ticket_error
86103
end
87104
end
105+
puts "#{GREEN}All commit messages match the guidelines!#{RESET}"
88106
end

acceptance/GHA_ACCEPTANCE.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Github Actions Acceptance Workflow
2+
3+
[acceptance.yaml](.github/workflows/acceptance.yaml)
4+
5+
This is a manual pipeline that can be triggered from the Actions tab
6+
of the GitHub repository. It runs the Beaker acceptance tests against
7+
either a pre-release build from
8+
https://artifacts.overlookinfratech.com, or released packages from the
9+
overlookinfratech.com repos.
10+
11+
The pipeline makes use of nested virtualization to standup an agent
12+
of a given OS platform for the runner to run Beaker against. A matrix
13+
strategy builds out jobs for different os/version combinations.
14+
15+
## Internals
16+
17+
The workflow makes use of a composite action currently living in a
18+
Bolt module
19+
[kvm_automation_tooling](https://github.com/jpartlow/kvm_automation_tooling/blob/main/action.yaml).
20+
21+
The action preps libvirt on the runner, installs Bolt and runs a
22+
[kvm_automation_tooling::standup_cluster](https://github.com/jpartlow/kvm_automation_tooling/blob/main/plans/standup_cluster.pp)
23+
plan which makes use of Terraform to generate a set of vms, set up
24+
ssh access and install the chose openvox-agent package.
25+
26+
Then the workflow sets up ruby/beaker per the acceptance/Gemfile and
27+
runs another
28+
[kvm_automation_tooling::dev::generate_beaker_hosts_file](https://github.com/jpartlow/kvm_automation_tooling/blob/main/plans/dev/generate_beaker_hosts_file.pp)
29+
plan to set up the beaker hosts file. And then runs beaker.
30+
31+
Note acceptance/Rakefile is not used in this workflow. That process
32+
relies on tasks provided by beaker-puppet which are tied to Perforce
33+
internals and repositories.
34+
35+
Instead the kvm_automation_tooling plans installs openvox-agent
36+
packages using another Bolt module
37+
[openvox_bootstrap](https://github.com/jpartlow/openvox_bootstrap/).
38+
This module has an install task which servers the same purpose as the
39+
puppet_agent module install task: initial installation of the agent.
40+
The openvox_bootstrap module is configured to use the voxpupuli
41+
repositories.
42+
43+
The module also has an
44+
[openvox_bootstrap::install_build_artifact](https://github.com/jpartlow/openvox_bootstrap/blob/main/tasks/install_build_artifact.json)
45+
task, which is what is used to install pre-release builds for testing
46+
in the pipeline.

acceptance/config/aio/options.rb

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,3 @@
11
{
22
:type => 'aio',
3-
:post_suite => [
4-
'teardown/common/099_Archive_Logs.rb',
5-
],
63
}
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Ensures that the /root/.ssh/environment file is
2+
# set up with a path that includes /opt/puppetlabs/bin.
3+
# Normally this would be called as part of beaker-puppet
4+
# install steps, but we are installing openvox-agent outside
5+
# of beaker-puppet, since it doesn't handle openvox.
6+
test_name('configure root ssh environment path') do
7+
configure_type_defaults_on(agents)
8+
end

acceptance/tests/ensure_facter_command_can_be_executed.rb

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,9 @@
22

33
require 'puppet/acceptance/common_utils'
44

5-
facter = agent['platform'] =~ /win/ ? 'cmd /c facter' : 'facter'
6-
75
agents.each do |agent|
86
step "test facter command" do
7+
facter = agent['platform'] =~ /win/ ? 'cmd /c facter' : 'facter'
98
on agent, "#{facter} --version", :acceptable_exit_codes => [0]
109
end
1110
end

0 commit comments

Comments
 (0)