Deploying of OCP4.6 with faros failing

[abhilashopGit]

Hi Ryan,

I am trying to deploy OCP4.6 with faros, and i created following link for farosctl to pin the version to 4.6.

[farosadmin@bastion bin]$ pwd
/home/farosadmin/bin
[farosadmin@bastion bin]$ ll farosctl.4.6
lrwxrwxrwx. 1 farosadmin farosadmin 8 Aug 23 10:17 farosctl.4.6 -> farosctl
[farosadmin@bastion bin]$

And deploying the cluster with following command,

[farosadmin@bastion ~]$ farosctl.4.6 install-plan cluster

The deployment is stuck at following step,

---

|
|-- create cluster
|

STEP 1: PERFORM PRE-INSTALL CHECKS
STEP 2: ENSURE ALL CLUSTER NODES ARE POWERED DOWN
management : power off kvm server power state 192.168.139.1>bootstrap
STEP 3: ENSURE ALL CLUSTER NODES ARE POWERED DOWN
management : power off ilo server power state blade-5
STEP 4: ENSURE ALL CLUSTER NODES ARE POWERED DOWN
management : power off ilo server power state blade-1
STEP 5: ENSURE ALL CLUSTER NODES ARE POWERED DOWN
management : power off ilo server power state blade-3
STEP 6: CREATE THE CONTROL PLANE
management : network boot ilo servers blade-1 blade-3 blade-5
STEP 7: CREATE THE BOOTSTRAP NODE
management : network boot kvm servers 192.168.139.1>bootstrap
wait for bootstrap to finish provisioning:

And when i check the bootstap vm console, i see following error.

Displaying logs from failed units: ignition-fetch-offline.service
-- Logs begin at Tue 2021-08-24 16:47:54 UTC, end at Tue 2021-08-24 16:47:56 UTC. --
Aug 24 16:47:56 ignition[680]: parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e
Aug 24 16:47:56 systemd[1]: Failed to start Ignition (fetch-offline).
Aug 24 16:47:56 ignition[680]: parsed url from cmdline: ""
Aug 24 16:47:56 ignition[680]: no config URL provided
Aug 24 16:47:56 ignition[680]: reading system config file "/usr/lib/ignition/user.ign"
Aug 24 16:47:56 ignition[680]: parsing config with SHA512: e9bd0629fae3427c71482de3f5486aceb0d7bb135ae21b4673442fcb26076a7699b1db2272f57f16c9c3bafba6cb15881a033999a4c77131b797182c114c18dc
Aug 24 16:47:56 ignition[680]: failed to fetch config: unsupported config version
Aug 24 16:47:56 ignition[680]: failed to acquire config: unsupported config version
Aug 24 16:47:56 systemd[1]: ignition-fetch-offline.service: Triggering OnFailure= dependencies.
Aug 24 16:47:56 ignition[680]: Ignition failed: unsupported config version
Press Enter for emergency shell or wait 0 minutes 30 seconds for reboot.

Please let me know what could be the reason for this error on bootstrap node ?

[abhilashopGit]

I executed the following commands to restart the cluster install and the behavior is different now,

farosctl apply
farosctl create install-repos
farosctl create cluster

Now the installation is hanging at following step,

STEP 7: CREATE THE BOOTSTRAP NODE
management : network boot kvm servers 192.168.139.1>bootstrap
STEP 8: WAIT FOR THE OPENSHIFT INSTALLATION TO COMPLETE
wait for bootstrap to complete:

And looks like bootstarp vm doesn't show any error,

Red Hat Enterprise Linux CoreOS 47.83.202103251640-0 (Ootpa) 4.7
SSH host key: SHA256:1u8FmuMt7vxaf+3czmmH01DT4QA133OaVZH1eWT+HZU (ECDSA)
SSH host key: SHA256:jCm2MCwJmFF3UUgQUEgCSO3T5+Vnh8PdOnIe9gN6XB4 (ED25519)
SSH host key: SHA256:8DCbXlZkohV4DJutow/drMlB6RYa50f0oiYqrqlWU5E (RSA)
enp1s0: 192.168.139.3 fe80::5054:ff:fe70:680e
bootstrap login:

And it fails with following error message,

STEP 8: WAIT FOR THE OPENSHIFT INSTALLATION TO COMPLETE
wait for bootstrap to complete: [WARNING]: Error deleting remote temporary files (rc: 1, stderr: })
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib64/python3.8/logging/init.py", line 1085, in emit
self.flush()
File "/usr/lib64/python3.8/logging/init.py", line 1065, in flush
self.stream.flush()
PermissionError: [Errno 13] Permission denied
Call stack:
File "/deps/python/bin/ansible-playbook", line 135, in
exit_code = cli.run()
File "/deps/python/lib/python3.8/site-packages/ansible/cli/playbook.py", line 137, in run
results = pbex.run()
File "/deps/python/lib/python3.8/site-packages/ansible/executor/playbook_executor.py", line 189, in run
result = self._tqm.run(play=play)
File "/deps/python/lib/python3.8/site-packages/ansible/executor/task_queue_manager.py", line 315, in run
play_return = strategy.run(iterator, play_context)
File "/deps/python/lib/python3.8/site-packages/ansible/plugins/strategy/linear.py", line 315, in run
self._queue_task(host, task, task_vars, play_context)
File "/deps/python/lib/python3.8/site-packages/ansible/plugins/strategy/init.py", line 392, in _queue_task
worker_prc.start()
File "/deps/python/lib/python3.8/site-packages/ansible/executor/process/worker.py", line 100, in start
return super(WorkerProcess, self).start()
File "/usr/lib64/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/usr/lib64/python3.8/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/usr/lib64/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/usr/lib64/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/deps/python/lib/python3.8/site-packages/ansible/executor/process/worker.py", line 134, in run
return self._run()
File "/deps/python/lib/python3.8/site-packages/ansible/executor/process/worker.py", line 167, in _run
executor_result = TaskExecutor(
File "/deps/python/lib/python3.8/site-packages/ansible/executor/task_executor.py", line 158, in run
res = self._execute()
File "/deps/python/lib/python3.8/site-packages/ansible/executor/task_executor.py", line 582, in _execute
result = self._handler.run(task_vars=variables)
File "/deps/python/lib/python3.8/site-packages/ansible/plugins/action/shell.py", line 25, in run
result = command_action.run(task_vars=task_vars)
File "/deps/python/lib/python3.8/site-packages/ansible/plugins/action/command.py", line 30, in run
self._remove_tmp_path(self._connection._shell.tmpdir)
File "/deps/python/lib/python3.8/site-packages/ansible/plugins/action/init.py", line 456, in _remove_tmp_path
display.warning('Error deleting remote temporary files (rc: %s, stderr: %s})'
File "/deps/python/lib/python3.8/site-packages/ansible/utils/display.py", line 409, in warning
self.display(new_msg, color=C.COLOR_WARN, stderr=True)
File "/deps/python/lib/python3.8/site-packages/ansible/utils/display.py", line 312, in display
logger.log(lvl, msg2)
Message: '[WARNING]: Error deleting remote temporary files (rc: 1, stderr: })\n'
Arguments: ()
wait for bootstrap to complete
failed: localhost: failed to transfer file to /data/.ansible_tmp/ansible-tmp-1629825691.3431652-559-182355215220861/AnsiballZ_command.py: [Errno 13] Permission denied: b'/data/.ansible_tmp/ansible-tmp-1629825691.3431652-559-182355215220861/AnsiballZ_command.py'
there was an error during install
failed: localhost: {'_ansible_no_log': False, 'changed': False}

create cluster Failed