Failure when trying to reproduce the README instructions

[zyt755]

Hi, I follow the instruction to install revng. But orchestra cannot work correctly to install rev.ng https://github.com/revng/orchestra/tree/master#building-from-source.

I type orc update and get

[+] WARNING - The remote HEADs cache does not exist, you should run `orchestra update`
[+] INFO - Updating orchestra configuration
[+] INFO - Updating binary archives
0archives [00:00, ?archives/s]
[+] INFO - Resetting ls-remote cached info
[+] INFO - Updating ls-remote cached info
[+] INFO -                                      
Could not find the following repositories in any remote:
  - vscode
  - llvm-project
  - revng-qa
  - revng
  - revng-c
  - revng-vscode-theme
  - llvmcpy
  - revng-ui
  - qemu

You will not be able to install components that depend on them.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 931.15repository/s]

I guess it does not matter because revng is built from source. So I ignore it and continue the next step.
I type orc install --test revng and get error

ytt: Error: 
- assert.fail: fail: You must provide a data value for remote_base_urls in datavalue lib/util.lib.yml:9 | #@       
- assert.fail("You must provide a data value for " + attrname) in <top-level> components.yml:41 | remote_base_urls: #@ datavalue("remote_base_urls")

Because build from source, I comment variable remote_base_urls and binary_archives and get return

ytt: Error: Overlaying (in following order: components/boost.yml, components/bzip2.yml, components/clang-release.yml, components/cmake.yml, components/cython.yml, components/early-gcc.yml, components/elfutils.yml, components/expat.yml, components/gcc-runtime.yml, components/glib.yml, components/glibc.yml, components/gmp.yml, components/icu.yml, components/ida.yml, components/libX.yml, components/libb2.yml, components/libedit.yml, components/libffi.yml, components/libgcc.yml, components/libltdl.yml, components/libunwind.yml, components/libxcrypt.yml, components/libyaml.yml, components/lit.yml, components/llvm-documentation.yml, components/llvm.yml, components/llvmcpy.yml, components/meson.yml, components/mpc.yml, components/mpfr.yml, components/ncurses.yml, components/ninja.yml, components/nodejs-support.yml, components/nodejs.yml, components/openssl.yml, components/python-base-packages.yml, components/python.yml, components/pyyaml.yml, components/qemu.yml, components/readline.yml, components/revng-c.yml, components/revng-development.yml, components/revng-distributable.yml, components/revng-js_prerequisites.yml, components/revng-python-dependencies.yml, components/revng-qa-python-dependencies.yml, components/revng-qa.yml, components/revng-test.yml, components/revng-theme.yml, components/revng-ui.yml, components/revng-vscode-web.yml, components/revng.yml, components/source-highlight.yml, components/sqlite.yml, components/tcl.yml, components/toolchain/arch/aarch64.yml, components/toolchain/arch/arm.yml, components/toolchain/arch/host.yml, components/toolchain/arch/i386.yml, components/toolchain/arch/mips.yml, components/toolchain/arch/mipsel.yml, components/toolchain/arch/s390x.yml, components/toolchain/arch/win32.yml, components/toolchain/arch/win64.yml, components/toolchain/arch/x86_64.yml, components/toolchain/host-toolchain.yml, components/toolchain/macos/clang.yml, components/toolchain/macos/ld64.yml, components/toolchain/macos/libtapi.yml, components/toolchain/macos/sdk.yml, components/toolchain/msvc.yml, components/vscode-web.yml, components/xz.yml, components/zlib.yml, lib/overlays/build-from_source.yml):
  Document on line components/boost.yml:55:
    Expected number of matched nodes to be 1, but was 2 (lines: components.yml:1, user_options.yml:1)

So, I delete components.yml and leave user_options.yml.
user_options.yml contains

#@overlay/replace    
build_from_source:   
      - revng

and get return

[+] ERROR - Invalid configuration. Got the following error at path $:
Additional properties are not allowed ('build_from_source' was unexpected)

Could you tell me how to install revng? I guess the guide of revng installation has something wrong.

[aleclearmind]

Your user_options.yml should look like this:

#! This file was automatically generated by orchestra
#! Edit it to suit your preferences

#@data/values
---
#@overlay/match missing_ok=True
remote_base_urls:
  - origin: "https://github.com/revng"

#@overlay/match missing_ok=True
binary_archives:
  - origin: "https://rev.ng/gitlab/revng/binary-archives.git"

#@overlay/replace
build_from_source:
  - revng

Can you try it and, if it works, point out what part of the documentation led you to a different result?

[zyt755]

I copy your user_options.yml, but still cannot install revng. I guess there is something wrong with version control. I fell confused why it is so complex to install revng.

(revng) a@Ubuntu:~/orchestra/.orchestra/config$ orc update
[+] INFO - Updating orchestra configuration
[+] INFO - Updating binary archives
[+] INFO - Trying to clone binary archive from remote origin (https://rev.ng/gitlab/revng/binary-archives.git)                                                                                     
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:06<00:00,  6.93s/archives, origin]
[+] INFO - Resetting ls-remote cached info
[+] INFO - Updating ls-remote cached info
[+] ERROR -                                                                                                                                                                                        
Could not git pull --ff-only the following repositories:
  - orchestra configuration (/home/atomu/Data/ubuntu/orchestra/.orchestra)

Suggestions:
    - check your network connection
    - commit your work
    - `git pull --rebase`, to pull remote changes and apply your commits on top
    - `git push` your changes to the remotes
[+] INFO -                                                                                                                                                                                         
Could not find the following repositories in any remote:
  - revng-c
  - revng-ui
  - qemu
  - llvmcpy
  - revng-vscode-theme
  - revng
  - vscode
  - revng-qa
  - llvm-project

You will not be able to install components that depend on them.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:09<00:00,  1.08s/repository]
(revng) a@Ubuntu:~/orchestra/.orchestra/config$ orc install --test revng
[+] ERROR - HEAD commit for revng not available. Run `orc update`.
(revng) a@Ubuntu:~/orchestra/.orchestra/config$ orc update
[+] INFO - Updating orchestra configuration
[+] INFO - Updating binary archives
  0%|                                                                                                                                                          | 0/1 [00:00<?, ?archives/s, origin]
[+] ERROR - Internal subprocess failed with exit code 128:
git clean -d --force
Output stream:
fatal: detected dubious ownership in repository at '/home/a/orchestra/.orchestra/binary-archives/origin'
To add an exception for this directory, call:

        git config --global --add safe.directory /home/a/orchestra/.orchestra/binary-archives/origin

[aleclearmind]

Can you paste orc --log-level=DEBUG update?

[zyt755]

I follow your instruction. Finally, I successfully installed revng. THX!
However, a cd back error is detected.

Pass bash-check failed
  In typescript/build-model.sh line 31:                                                                                                                                                            
  cd ..                                                                                           
  ^-- SC2103: Use a ( subshell ) to avoid having to cd back.    
[+] ERROR - Script failed with exit code 1

typescript/revng/build-model.sh line 31
should be replaced by a subshell

(
cd model.ts-package

cp "$1/package-model.json" package.json
cp "$1/tsconfig-model.json" tsconfig.json
cp "$1/tuple_tree.ts" tuple_tree.ts
cp ../lib/typescript/model.ts .
CHECKSUM=$(cat model.ts tuple_tree.ts | sha1sum - | cut -d' ' -f1)
sed -i "s;##CHECKSUM##;$CHECKSUM;g" package.json
cp -rT "$2" node_modules
./node_modules/.bin/tsc -p .
npm pack --silent > /dev/null
cp revng-model-1.*.tgz ../model.ts.tgz
)

[mrjackv]

@zyt755 What is the version reported by shellcheck --version?
I suspect there might be a mismatch

[zyt755]

Thx guys!
@aleclearmind I think this user_options.yml confuses me.

@mrjackv shellcheck version: 0.4.6

[mrjackv]

@mrjackv shellcheck version: 0.4.6

That's a quite old version (Link)
I'd suggest to upgrade it to at least 0.8.0, which is the version used in Ubuntu 22.04, which is what we use in the CI

[aleclearmind]

@aleclearmind I think this user_options.yml confuses me.

It clearly states:

If you have access to rev.ng GitLab, in order to access private components, your configuration should be similar to the following:

But yeah, maybe we should put in the title something more explicit about the fact that "rev.ng developers" means "developers with access to our private internal repos".

[zyt755]

@mrjackv shellcheck version: 0.4.6

That's a quite old version (Link) I'd suggest to upgrade it to at least 0.8.0, which is the version used in Ubuntu 22.04, which is what we use in the CI

OK! I update it! Thx very much!

[zyt755]

@aleclearmind I think this user_options.yml confuses me.

It clearly states:

If you have access to rev.ng GitLab, in order to access private components, your configuration should be similar to the following:

But yeah, maybe we should put in the title something more explicit about the fact that "rev.ng developers" means "developers with access to our private internal repos".

Yeah, I misunderstand the docs. I think ordinary users can only see what is allowed to be seen. Otherwise, this content will only be spread between developers.