diff --git a/README.md b/README.md index 84856a9..39cc3be 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # HolyLabRegistry -This registry allows you to use packages from HolyLab in Julia 0.7/1.x. +This registry allows you to use packages from HolyLab in 1.x. (For very old Julia versions, see notes at the bottom.) # Usage @@ -12,8 +12,6 @@ If you're using at least Julia 1.1, then you can add this registry with (The `]` enters Pkg mode when you type it at the REPL prompt, see https://docs.julialang.org/en/v1/stdlib/Pkg/.) -For earlier Julia versions, manually `git clone` this repository under `DEPOT_PATH/registries`. (Usually, `DEPOT_PATH = /home/username/.julia`) - Then, we can use lab private packages (or unregistered public ones) as if they are registered ones. # To use git protocol in GitHub @@ -23,98 +21,108 @@ For windows users, you can get some information at https://gist.github.com/bsara 0. Specific preparation for Windows - - Create a folder at the root of your user home folder (Example: C:/Users/uname/) called .ssh. - - Create the following files if they do not already exist (paths begin from the root of your user home folder): - - .ssh/config
- .bash_profile
- .bashrc
+ - Create a folder at the root of your user home folder (Example: C:/Users/uname/) called .ssh. + - Create the following files if they do not already exist (paths begin from the root of your user home folder): + + .ssh/config
+ .bash_profile
+ .bashrc
1. Create a New SSH Key - 1.1 Generating a new SSH key at a local machine. - - Open git bash and paste text below, substituting in your GitHub email address. - ``` - $ ssh-keygen -t ecdsa -b 521 -C "your_email@example.com" - ``` - Note: Around September 1, 2021, GitHub has added new security requirements for newly added RSA keys. Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information. - - - - - - When you're prompted to "Enter a file in which to save the key," press Enter. This accepts the default file location. - ``` - Enter a file in which to save the key (/home/you/.ssh/id_ecdsa): [Press enter] - ``` + Most interactions with GitHub require authentication, which is handled by SSH. There are at least two distinct cases: - - At the prompt, type a secure passphrase if you want. - ``` - Enter passphrase (empty for no passphrase): [Type a passphrase] - Enter same passphrase again: [Type passphrase again] - ``` + - `github_push_key`: when you want to `push` code from a local machine up to GitHub: for this case, your private key remains only on your local machine and your public key needs to be registered with your GitHub account. + - `github_CI_key`: when you want a package you're developing and hosting on GitHub to be able to access private repositories (e.g., during CI): for this case, the private key gets pasted into a repository secret, and the public key needs to be registered with your GitHub account. - 1.2 Adding your SSH key to the ssh-agent - - Start the ssh-agent in the background. - ``` - $ eval "$(ssh-agent -s)" - Agent pid 59566 - ``` + If both of these uses apply to you, you should generate two separate keys for the two cases. - - Add your SSH private key to the ssh-agent - ``` - $ ssh-add ~/.ssh/id_ecdsa - ``` -2. Setup SSH Authentication for Git Bash on Windows (Safe to skip for Linux) + Here are the steps needed to generate a key: - 2.1 Configure SSH for Git Hosting Server - - Add the following text to .ssh/config (.ssh should be found in the root of your user home folder): - ``` - Host github.com
- Hostname github.com
- IdentityFile ~/.ssh/id_ecdsa - ``` - - 2.2 Enable SSH Agent Startup Whenever Git Bash is Started - - First, ensure that following lines are added to .bash_profile, which should be found in your root user home folder: - ``` - test -f ~/.profile && . ~/.profile - test -f ~/.bashrc && . ~/.bashrc - ``` - - Now, add the following text to .bashrc, which should be found in your root user home folder: - ``` - # Start SSH Agent - #---------------------------- + 1.1 Generating a new SSH key at a local machine. - SSH_ENV="$HOME/.ssh/environment" + - Open git bash and paste text below, substituting in your GitHub email address. + ``` + $ ssh-keygen -t ecdsa -b 521 -C "your_email@example.com" + ``` + Note: Around September 1, 2021, GitHub has added new security requirements for newly added RSA keys. Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information. - function run_ssh_env { - . "${SSH_ENV}" > /dev/null - } + - When you're prompted to "Enter a file in which to save the key," press Enter. This accepts the default file location. + ``` + Enter a file in which to save the key (/home/you/.ssh/id_ecdsa): [Press enter] + ``` + It's recommended to name the key something informative, e.g., `github_push_key` or `github_CI_key` for the two cases described at the top of this section. - function start_ssh_agent { - echo "Initializing new SSH agent..." - ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}" - echo "succeeded" - chmod 600 "${SSH_ENV}" + - At the prompt, type a secure passphrase if you want. (Do not use a passphrase for `github_CI_key` cases, just hit at the prompts below.) + ``` + Enter passphrase (empty for no passphrase): [Type a passphrase] + Enter same passphrase again: [Type passphrase again] + ``` - run_ssh_env; + 1.2 Adding your SSH key to the ssh-agent (only for `github_push_key`) - ssh-add ~/.ssh/id_ecdsa; - } + - Start the ssh-agent in the background. + ``` + $ eval "$(ssh-agent -s)" + Agent pid 59566 + ``` - if [ -f "${SSH_ENV}" ]; then - run_ssh_env; - ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || { - start_ssh_agent; + - Add your SSH private key to the ssh-agent + ``` + $ ssh-add ~/.ssh/id_ecdsa + ``` + + + +2. Setup SSH Authentication for Git Bash on Windows (Safe to skip for Linux) (Only for `github_push_key`) + + 2.1 Configure SSH for Git Hosting Server + Add the following text to .ssh/config (.ssh should be found in the root of your user home folder): + ``` + Host github.com
+ Hostname github.com
+ IdentityFile ~/.ssh/id_ecdsa + ``` + + 2.2 Enable SSH Agent Startup Whenever Git Bash is Started + - First, ensure that following lines are added to .bash_profile, which should be found in your root user home folder: + ``` + test -f ~/.profile && . ~/.profile + test -f ~/.bashrc && . ~/.bashrc + ``` + - Now, add the following text to .bashrc, which should be found in your root user home folder: + ``` + # Start SSH Agent + #---------------------------- + + SSH_ENV="$HOME/.ssh/environment" + + function run_ssh_env { + . "${SSH_ENV}" > /dev/null } - else - start_ssh_agent; - fi - ``` - -3. Adding a new SSH key to your GitHub account + + function start_ssh_agent { + echo "Initializing new SSH agent..." + ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}" + echo "succeeded" + chmod 600 "${SSH_ENV}" + + run_ssh_env; + + ssh-add ~/.ssh/id_ecdsa; + } + + if [ -f "${SSH_ENV}" ]; then + run_ssh_env; + ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || { + start_ssh_agent; + } + else + start_ssh_agent; + fi + ``` + +3. Adding a new SSH key to your GitHub account: - Copies the contents of the id_ecdsa.pub file in the local machine to your clipboard - Go to GitHub site - In the upper-right corner of any page, click your profile photo, then click Settings. @@ -148,14 +156,14 @@ You have two options: - Using [PkgTemplates.jl](https://github.com/invenia/PkgTemplates.jl) (recommended): - To create a new package and host it in your own GitHub account, use + To create a new package and host it the HolyLab organization account, use ```julia julia> using PkgTemplates - julia> t = Template(ssh=true, plugins=[TravisCI()]) # creates a template for your personal account + julia> tpl = Template(user="HolyLab"; plugins=[GitHubActions(), Codecov(), Documenter{GitHubActions}()]) Template: - → User: timholy + → User: HolyLab → Host: github.com → License: MIT (Tim Holy 2019) → Package directory: /tmp/pkgs/dev @@ -164,15 +172,11 @@ You have two options: → Commit Manifest.toml: No → Plugins: None - julia> generate("MyPkg", t) + julia> tpl("MyPkg") # lots of output ``` - If you plan to host your new package on `HolyLab`, instead use - - ```julia - julia> t = Template(user="HolyLab", ...) - ``` + If you instead want to host your package within your personal GitHub account, omit `user="HolyLab"` from the above `Template` call (or specify your own user name). ### Adding dependent packages @@ -243,9 +247,12 @@ $ git branch -D teh/SomeNewPkg 3bd9afcd-55df-531a-9b34-dc642dce7b95 = { name = "RFFT", path = "RFFT" } ``` -## Accessing the HolyLabRegistry in a CI workflow +## Accessing HolyLabRegistry or private packages in CI tests -This is required only if your package uses other packages which are registered in this HolyLabRegistry registry. +This is required only if your package uses other packages which are registered in this HolyLabRegistry registry, +or your package depends on unregistered private packages. + +- In the repository's `Settings`, go to `Secrets and Variables` and expand the caret, click on `Actions`. Add a `New repository secret` called `SSH_PRIVATE_KEY`, and copy the contents of the *private* key for your `github_CI_key` (i.e., not the file ending in `.pub`) - Set a secret key in the repository settings if you want to use private dependent packages in our lab. The key should not include a passphrase when the key pair is generated. Add the public key as a 'new SSH key' @@ -263,8 +270,8 @@ jobs: strategy: # setting ... steps: - - uses: actions/checkout@v3 - - name: Setup SSH Keys and known_hosts # This section is required if the dependent packages include private packages in our lab. + - uses: actions/checkout@v3 + - name: Setup SSH Keys and known_hosts # This section is required if the dependent packages include private packages in our Lab. env: SSH_AUTH_SOCK: /tmp/ssh_agent.sock run: | @@ -287,7 +294,25 @@ jobs: - uses: julia-actions/julia-runtest@v1 ``` -## Tagging a new release +If you're instead wanting to manually add unregistered dependencies, then you should have something like this: + +``` + - name: Build package manually # manual because of dependency on MyPrivatePackage + run: | + using Pkg + Pkg.Registry.add() # add Julia's General registry + Pkg.develop(url="git@github.com:HolyLab/MyPrivatePackage.jl.git") + Pkg.build() + shell: julia --color=yes --project="@." {0} + env: + GITHUB_TOKEN: ${{ github.token }} + SSH_AUTH_SOCK: /tmp/ssh_agent.sock + JULIA_PKG_PRECOMPILE_AUTO: "no" +``` + +These lines replace `julia-actions/julia-buildpkg@v1`, so do not use this action if you're doing things this way. + +## Registering a new release in HolyLabRegistry ### In the package directory @@ -325,3 +350,9 @@ Use the sha from the `git cat-file` command above. # See also - Creating a registry : https://discourse.julialang.org/t/creating-a-registry/12094 + +# Tips for Julia 0.7 and 1.0 + +In case of very old code, here are some tips: + +- `pkg> registry add` does not work. For earlier Julia versions, manually `git clone` this repository under `DEPOT_PATH/registries`. (Usually, `DEPOT_PATH = /home/username/.julia`)