Skip to content

Upgrade netCDF-C stack #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lesserwhirls merged 2 commits into from
Mar 10, 2025
Merged

Upgrade netCDF-C stack #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a14b3df
Remove aws vars for netCDF-C build
lesserwhirls Mar 7, 2025
e04a503
Upgrade netCDF-C
lesserwhirls Mar 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 24 additions & 24 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,10 @@ We use the following roles when provisioning our images:

### netCDF-C
* location: `/usr/thredds-test-environment`
* version: `4.8.1`
* version: `4.9.2`
* dependencies (same location):
* zlib version: `1.2.11`
* hdf5 version: `1.12.1`
* zlib version: `1.3.1`
* hdf5 version: `1.14.6`

### maven:
* location: `/usr/thredds-test-environment/mvn`
Expand Down Expand Up @@ -155,26 +155,26 @@ We use the following roles when provisioning our images:
### Docker Image

~~~
docker.docker-jenkins: Tuesday 25 February 2025 15:28:53 +0000 (0:00:02.993) 0:25:00.302 ******
docker.docker-jenkins: Saturday 08 March 2025 01:34:12 +0000 (0:00:02.971) 0:25:25.026 ********
docker.docker-jenkins: ===============================================================================
docker.docker-jenkins: Wait for the HDF5 async test task to complete. ------------------------ 362.87s
docker.docker-jenkins: libnetcdf-and-deps : Install hdf5. ------------------------------------ 328.13s
docker.docker-jenkins: libnetcdf-and-deps : Configure netCDF-c. ------------------------------ 180.21s
docker.docker-jenkins: libnetcdf-and-deps : Configure hdf5. ---------------------------------- 135.15s
docker.docker-jenkins: temurin : Fetch latest Temurin Java builds. ---------------------------- 83.71s
docker.docker-jenkins: zulu : Fetch latest Zulu Java builds. ---------------------------------- 77.12s
docker.docker-jenkins: general-packages : Install os managed tools. --------------------------- 41.00s
docker.docker-jenkins: corretto : Fetch latest Corretto Java builds. -------------------------- 40.41s
docker.docker-jenkins: zulu : Unpack Zulu Java Installations. --------------------------------- 37.98s
docker.docker-jenkins: general-packages : Install os managed tools. --------------------------- 36.87s
docker.docker-jenkins: libnetcdf-and-deps : Install netCDF-c. --------------------------------- 34.90s
docker.docker-jenkins: corretto : Unpack Corretto Java Installations. ------------------------- 28.92s
docker.docker-jenkins: temurin : Unpack Temurin Java Installations. --------------------------- 28.57s
docker.docker-jenkins: general-packages : Install os managed tools. --------------------------- 15.22s
docker.docker-jenkins: libnetcdf-and-deps : Install zlib. -------------------------------------- 6.27s
docker.docker-jenkins: security : Update SSH configuration to be more secure. ------------------ 5.02s
docker.docker-jenkins: libnetcdf-and-deps : Configure zlib. ------------------------------------ 4.52s
docker.docker-jenkins: zulu : Read versions of installed Zulu. --------------------------------- 3.74s
docker.docker-jenkins: libnetcdf-and-deps : Download and unpack hdf5. -------------------------- 3.65s
docker.docker-jenkins: cleanup : Remove packages that are not needed in final environment. ----- 3.54s
docker.docker-jenkins: Wait for the HDF5 async test task to complete. ------------------------ 483.06s
docker.docker-jenkins: libnetcdf-and-deps : Make hdf5. --------------------------------------- 327.68s
docker.docker-jenkins: libnetcdf-and-deps : Configure netCDF-c. ------------------------------ 174.98s
docker.docker-jenkins: libnetcdf-and-deps : Configure hdf5. ---------------------------------- 148.53s
docker.docker-jenkins: temurin : Fetch latest Temurin Java builds. ---------------------------- 53.85s
docker.docker-jenkins: zulu : Unpack Zulu Java Installations. --------------------------------- 39.85s
docker.docker-jenkins: general-packages : Install os managed tools. --------------------------- 39.50s
docker.docker-jenkins: general-packages : Install os managed tools. --------------------------- 34.55s
docker.docker-jenkins: libnetcdf-and-deps : Install netCDF-c. --------------------------------- 32.28s
docker.docker-jenkins: corretto : Unpack Corretto Java Installations. ------------------------- 28.85s
docker.docker-jenkins: temurin : Unpack Temurin Java Installations. --------------------------- 28.05s
docker.docker-jenkins: zulu : Fetch latest Zulu Java builds. ---------------------------------- 27.61s
docker.docker-jenkins: corretto : Fetch latest Corretto Java builds. -------------------------- 16.80s
docker.docker-jenkins: general-packages : Install os managed tools. --------------------------- 11.56s
docker.docker-jenkins: libnetcdf-and-deps : Download and unpack hdf5. -------------------------- 9.03s
docker.docker-jenkins: libnetcdf-and-deps : Install zlib. -------------------------------------- 6.08s
docker.docker-jenkins: security : Update SSH configuration to be more secure. ------------------ 5.03s
docker.docker-jenkins: libnetcdf-and-deps : Configure zlib. ------------------------------------ 4.44s
docker.docker-jenkins: zulu : Read versions of installed Zulu. --------------------------------- 4.24s
docker.docker-jenkins: cleanup : Remove packages that are not needed in final environment. ----- 3.46s
~~~
9 changes: 7 additions & 2 deletions packer/provisioners/ansible/roles/libnetcdf-and-deps/tasks/hdf5.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
- name: Download and unpack hdf5.
unarchive:
src: "https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-{{ hdf5_version_major }}.{{ hdf5_version_minor }}/hdf5-{{ hdf5_version_full }}/src/hdf5-{{ hdf5_version_full }}.tar.gz"
src: "https://github.com/HDFGroup/hdf5/releases/download/hdf5_{{ hdf5_version_full }}/hdf5-{{ hdf5_version_full }}.tar.gz"
dest: "{{ tmp_dir }}"
creates: "{{ hdf5_src_dir }}"
remote_src: yes
Expand All @@ -16,10 +16,15 @@
chdir: "{{ hdf5_src_dir }}"
creates: "{{ hdf5_src_dir }}/config.log"

- name: Install hdf5.
- name: Make hdf5.
command: "make -j{{ make_jobs}} install"
args:
chdir: "{{ hdf5_src_dir }}"

- name: Install hdf5.
command: "make install"
args:
chdir: "{{ hdf5_src_dir }}"
creates: "{{ install_dir }}/lib/libhdf5.so"

# Run after install for performance reasons, as check can take quite some time
Expand Down
2 changes: 1 addition & 1 deletion packer/provisioners/ansible/roles/libnetcdf-and-deps/tasks/netcdf-c.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
until: unarchive_result.failed == false

- name: Configure netCDF-c.
command: ./configure --prefix={{ install_dir }} --disable-dap --disable-utilities --disable-static
command: ./configure --prefix={{ install_dir }} --disable-dap --disable-utilities --disable-static --disable-nczarr --disable-libxml2 --disable-byterange
args:
chdir: "{{ netcdf_src_dir }}"
creates: "{{ netcdf_src_dir }}/config.log"
Expand Down
2 changes: 1 addition & 1 deletion packer/provisioners/ansible/roles/libnetcdf-and-deps/tasks/zlib.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
- name: Download and unpack zlib.
unarchive:
src: https://downloads.sourceforge.net/project/libpng/zlib/{{ zlib_version }}/zlib-{{ zlib_version }}.tar.gz
src: https://www.zlib.net/zlib-{{ zlib_version }}.tar.gz
dest: "{{ tmp_dir }}"
creates: "{{ zlib_src_dir }}"
remote_src: yes
Expand Down
3 changes: 0 additions & 3 deletions packer/provisioners/ansible/roles/libnetcdf-and-deps/vars/amazon-ebs.yml
View file
Open in desktop

This file was deleted.

8 changes: 4 additions & 4 deletions packer/provisioners/ansible/roles/libnetcdf-and-deps/vars/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
zlib_version: 1.2.11
zlib_version: 1.3.1
zlib_src_dir: "{{ tmp_dir }}/zlib-{{ zlib_version }}"

hdf5_version_major: 1
hdf5_version_minor: 12
hdf5_version_point: 1
hdf5_version_minor: 14
hdf5_version_point: 6
hdf5_version_full: "{{ hdf5_version_major }}.{{ hdf5_version_minor }}.{{ hdf5_version_point }}"
hdf5_src_dir: "{{ tmp_dir }}/hdf5-{{ hdf5_version_full }}"

netcdf_version: 4.8.1
netcdf_version: 4.9.2
netcdf_src_dir: "{{ tmp_dir }}/netcdf-c-{{ netcdf_version }}"