Skip to content

Commit 69afa9b

Browse files
authored
Merge pull request #228 from NCAR/release_v1.1
2 parents eaa1ff5 + f5b62d6 commit 69afa9b

File tree

122 files changed

+3190
-591
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

122 files changed

+3190
-591
lines changed

.github/workflows/documentation.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ jobs:
2626
python-version: '3.8'
2727
- name: Install dependencies
2828
run: |
29-
python -m pip install --upgrade sphinx sphinx-gallery sphinx_rtd_theme
29+
python -m pip install --upgrade sphinx sphinx-gallery sphinx_rtd_theme sphinx-design
3030
python -m pip install python-dateutil requests Pillow
3131
- name: Build docs
3232
run: ./.github/jobs/build_documentation.sh

.readthedocs.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,14 @@ version: 2
99
#formats: all
1010
formats: [pdf]
1111

12+
build:
13+
os: ubuntu-22.04
14+
tools:
15+
python: "3.10"
16+
1217
# Optionally set the version of Python and requirements required to build your
1318
# docs
1419
python:
15-
version: 3.8
1620
install:
1721
- requirements: docs/requirements.txt
1822

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# WRF Cloud
2-
[![codecov](https://codecov.io/github/ncar/wrfcloud/branch/develop/graph/badge.svg?token=1950DDI9D2)](https://codecov.io/github/ncar/wrfcloud)
2+
[![codecov](https://codecov.io/github/ncar/wrfcloud/branch/develop/graph/badge.svg?token=1950DDI9D2)](https://app.codecov.io/gh/NCAR/wrfcloud)
33

44
The WRF Cloud framework is a cloud-based forecasting system designed to facilitate easy, cost-effective, state-of-the-art numerical weather prediction system forecasts for communities that lack large computational resources.
55

66
## Documetation & Information
7-
System overview, installation and usage information can be found in the [Users Guide](https://wrfcloud.readthedocs.io/en/latest/).
7+
System overview, installation and usage information can be found in the [Users Guide](https://wrfcloud.readthedocs.io/en/stable/).
88

99
Questions can be posted to the [GitHub Discussions](https://github.com/NCAR/wrfcloud/discussions) page.
1010

codebuild/Dockerfile.wrfcloud-test

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,14 @@
11
FROM amazonlinux:latest
22

3+
# check the system architecture
4+
RUN if [[ "$(uname -m)" != "x86_64" ]]; then echo 'Must run on x86_64 architecture for compatibility with AWS codebuild'; exit 1; fi
5+
36
# install required os packages
47
RUN yum -y update
5-
RUN yum -y install zip gzip tar java-11-amazon-corretto git which procps-ng jq
8+
RUN yum -y install zip gzip tar java-11-amazon-corretto git which procps-ng jq wget
69

710
# download, build, and install python3
8-
RUN yum -y install gcc-c++ make openssl-devel bzip2-devel libffi-devel sqlite-devel
11+
RUN yum -y install gcc-c++ make openssl-devel bzip2-devel libffi-devel sqlite-devel zlib-devel
912
RUN mkdir -p /opt/src
1013
WORKDIR /opt/src
1114
RUN curl https://www.python.org/ftp/python/3.9.13/Python-3.9.13.tgz > Python-3.9.13.tgz
@@ -33,9 +36,11 @@ RUN echo "region = us-east-1" >> ~/.aws/credentials
3336
RUN echo "output = json" >> ~/.aws/credentials
3437

3538
# install nodejs and angular cli
36-
RUN curl --silent --location https://rpm.nodesource.com/setup_16.x | bash -
37-
RUN yum -y install nodejs
38-
RUN npm install -g @angular/cli
39+
RUN touch "${HOME}/.bashrc"
40+
RUN wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.2/install.sh | bash
41+
RUN source ~/.nvm/nvm.sh; nvm install 16
42+
RUN source ~/.bashrc; npm install -g @angular/cli@14
43+
RUN echo -n 'ewogICIkc2NoZW1hIjogIi4vbm9kZV9tb2R1bGVzL0Bhbmd1bGFyL2NsaS9saWIvY29uZmlnL3NjaGVtYS5qc29uIiwKICAidmVyc2lvbiI6IDEsCiAgImNsaSI6IHsKICAgICJjb21wbGV0aW9uIjogewogICAgICAicHJvbXB0ZWQiOiB0cnVlCiAgICB9LAogICAgImFuYWx5dGljcyI6ICJhY2UxOTcxNS01Y2Q5LTQyYTItODI2Mi1jYmVlNDFhOTNjNWMiCiAgfSwKICAicHJvamVjdHMiOiB7fQp9Cg==' | base64 -d > ~/.angular-config.json
44+
RUN echo -n 'CiMgTG9hZCBBbmd1bGFyIENMSSBhdXRvY29tcGxldGlvbi4Kc291cmNlIDwobmcgY29tcGxldGlvbiBzY3JpcHQpCg==' | base64 -d >> ~/.bashrc
45+
RUN source ~/.bashrc
3946

40-
# start the local dynamodb service when the container starts
41-
#CMD java -jar /opt/dynamodb/DynamoDBLocal.jar -inMemory &

codebuild/buildspec.yaml

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
version: 0.1
1+
version: 0.2
22

33
phases:
44
pre_build:
@@ -9,22 +9,25 @@ phases:
99
build:
1010
commands:
1111
- echo "Installing package dependencies"
12-
- pip3 install $CODEBUILD_SRC_DIR/python/src
12+
- mv "${CODEBUILD_SRC_DIR}/python/src/setup.py" "${CODEBUILD_SRC_DIR}/python/src/setup-with-pygrib.py"
13+
- cat "${CODEBUILD_SRC_DIR}/python/src/setup-with-pygrib.py" | sed "s/'pygrib.*',//g" > "${CODEBUILD_SRC_DIR}/python/src/setup.py"
14+
- pip3 install "$CODEBUILD_SRC_DIR/python/src"
1315
- pip3 uninstall -y wrfcloud
1416
- echo "Checking for Python code errors"
15-
- pylint -E $CODEBUILD_SRC_DIR/python/src/wrfcloud
17+
- pylint -E "$CODEBUILD_SRC_DIR/python/src/wrfcloud"
1618
- echo "Running unit tests and code coverage"
17-
- $CODEBUILD_SRC_DIR/python/coverage.sh
18-
- cd $CODEBUILD_SRC_DIR/web
19+
- "$CODEBUILD_SRC_DIR/python/coverage.sh"
20+
- cd "$CODEBUILD_SRC_DIR/web"
21+
- source ~/.bashrc
1922
- npm install
2023
- ng build
2124
- cd dist/web/
22-
- zip -r ${CODEBUILD_SRC_DIR}/wrfcloud_web.zip .
25+
- zip -r "${CODEBUILD_SRC_DIR}/wrfcloud_web.zip" .
2326

2427
post_build:
2528
commands:
2629
- echo "Publishing code coverage report to codecov.io"
27-
- curl -s https://codecov.io/bash > codecov_temp.sh
30+
- curl -s https://codecov.io/bash > "${CODEBUILD_SRC_DIR}/codecov_temp.sh"
2831
- CODECOV_TOKEN=$(aws secretsmanager get-secret-value --secret-id CodeCovToken | jq -r .SecretString)
2932
- cd "${CODEBUILD_SRC_DIR}"
3033
- /bin/bash ./codecov_temp.sh -t ${CODECOV_TOKEN} -f ${CODEBUILD_SRC_DIR}/python/test/coverage.xml -R $CODEBUILD_SRC_DIR

codebuild/cf_codebuild.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ Resources:
138138
Auth:
139139
Type: OAUTH
140140
Resource: !Ref CodeBuildSourceCredential
141-
SourceVersion: feature-2/codebuild
141+
SourceVersion: develop
142142
TimeoutInMinutes: 120 # 2 hours
143143
Triggers:
144144
Webhook: true

docs/Users_Guide/administration.rst

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,5 @@ The reference ID can be used to search the CloudWatch Logs to find logs leading
143143

144144
Troubleshooting WRF Failures
145145
============================
146-
A full set of log files is saved when a WRF job either finishes or fails to complete. The log
147-
files are stored as a zip file in the system's S3 bucket. To find the zip file, look for
148-
`s3://wrfcloud-XXXXXXXX/jobs/<job-id>/logs.zip`
146+
Many log files are now available to view directly on the WRF Jobs status page of the user interface (see :numref:`wrf_jobs`). This is the best place to start looking for what caused a WRF job to fail. The full set of log files files are stored as a zip file in the system's S3 bucket. To find the zip file, look for `s3://wrfcloud-XXXXXXXX/jobs/<job-id>/logs.zip`
149147

docs/Users_Guide/faqs.rst

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
**************************
2+
Frequently Asked Questions
3+
**************************
4+
5+
.. dropdown:: How much does it cost?
6+
7+
The main costs include hosting a web domain, compute cluster time, and data egress. A web domain typically costs $12/year.
8+
The forecasts are run on AWS's hpc6a clusters that cost $3.00/hour. The data egress costs $0.09/GB. The total cost depends on user interaction with the forecast viewer. For example, to view every variable that is currently available for a 24-hour forecast with hourly output, the system uses about 700MB, which translates to about $0.06.
9+
10+
To give users a general idea of the range of costs, here are some example forecasts and their associated costs to compute. Note that these estimates are for running the forecast, which takes place on the most expensive part of the system: the compute nodes. Additional costs for tasks like viewing products are generally minimal and are described above.
11+
12+
+----------+---+-------+-----+----+-------+-----+------+
13+
| Name |Res|Grid |Fcst |Out |Wall |Cores|Cost |
14+
| | |Size |Len |Freq|Clock | | |
15+
+==========+===+=======+=====+====+=======+=====+======+
16+
| Caribbean|6km|700x500|24 hr|1 hr|2.17 hr|96 |$6.91 |
17+
+----------+---+-------+-----+----+-------+-----+------+
18+
| Brail |3km|500x520|24 hr|1 hr|1.62 hr|96 |$5.11 |
19+
+----------+---+-------+-----+----+-------+-----+------+
20+
| Denver |1km|250x250|24 hr|1 hr|1.53 hr|48 |$4.71 |
21+
+----------+---+-------+-----+----+-------+-----+------+
22+
23+
.. dropdown:: Can you run multiple domains?
24+
25+
Currently we support running single domains only. Multiple domains are on the list for future enhancements.
26+
27+
.. dropdown:: What is the input data used?
28+
29+
Initial conditions are provided by the Global Forecasting System (GFS) at 3-hourly time intervals. Future enhancements may allows for 1-hourly interval data to be used.
30+
31+
.. dropdown:: Can the input data source be changed?
32+
33+
No, at this time the system only uses the Global Forecasting System (GFS) for initial conditions. But future enhancements could allow for varying sources.
34+
35+
.. dropdown:: Can you download the model output?
36+
37+
At this time, the model data is not saved. Only the forecast plots viewable on the Forecast Viewer are saved. But, we expect to include the option to save model output in the next released version.
38+
39+
.. dropdown:: What cloud service providers are supported?
40+
41+
Currently the system only works on Amazon Web Services (AWS). Interested in porting it to another CSP? We are too! Let’s talk, send us an email.
42+
43+
.. dropdown:: What model is used?
44+
45+
The system is currently designed to use the WRF model.
46+
47+
.. dropdown:: A WRF job failed, how do I fix it?
48+
49+
WRF jobs can fail for many reasons. First, look at the log files for the failed job on the WRF Jobs page. The status of the job may help indicate which component has failed, e.g. metgrid, real, or wrf. In the :ref:`ui_log_viewer`, search the log files for that component first for a clue as to what went wrong. If the job is based off a new configuration (model domain, physics, etc.), it's possible that those settings aren't properly tuned. There are countless ways to configure the model. Properly tuning a new configuration requires some knowledge of WRF and NWP modeling. Refer to the `WRF documentation <https://www2.mmm.ucar.edu/wrf/users/docs/user_guide_v4/v4.4/contents.html>`_ for help.
50+
51+
Common errors you may encounter include:
52+
53+
* **Over-decomposition:** The error *"domain size is too small for this many processors"* or something similar may appear in the *Real* rsl.* files when the *Real* process fails. This means that too many processors were used for the domain size. If the *Set automatically* checkbox was selected for the Core Count in the model configuration GUI, it is possible that it overestimated. :ref:`Modify the configuration<ui_update_existing_config>` core count to uncheck *Set automatically*, manually set it to a smaller value, save, then :ref:`run a new forecast<run_wrf>` with the updated configuration.
54+
* **CFS errors:** "cfl" errors may occur in the *WRF* rsl.* log files. This means the model became unstable which can happen for a lot of reasons and often requires advanced debugging. The `WRF Forum <https://forum.mmm.ucar.edu/>`_ may have some clues.

docs/Users_Guide/graphics.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Current products available for viewing include:
1313

1414
* 2-meter Temperature
1515
* 2-meter Dew Point
16+
* 10-meter Wind Speed
1617
* 10-meter Wind (Speed/Direction vectors)
1718
* Mean Sea Level Pressure
1819
* 2-m Relative Humidity
@@ -27,6 +28,7 @@ Current products available for viewing include:
2728
* Temperature on Pressure Levels (1000 hPa, 925 hPa, 850 hPa, 700 hPa, 500 hPa, 300 hPa, 250 hPa, 100 hPa)
2829
* Relative Humidity on Pressure Levels (1000 hPa, 925 hPa, 850 hPa, 700 hPa, 500 hPa, 300 hPa, 250 hPa, 100 hPa)
2930
* Vertical Velocity on Pressure Levels (1000 hPa, 925 hPa, 850 hPa, 700 hPa, 500 hPa, 300 hPa, 250 hPa, 100 hPa)
31+
* Wind Speed on Pressure Levels (1000 hPa, 925 hPa, 850 hPa, 700 hPa, 500 hPa, 300 hPa, 250 hPa, 100 hPa)
3032
* Wind (Speed/Direction vectors) on Pressure Levels (1000 hPa, 925 hPa, 850 hPa, 700 hPa, 500 hPa, 300 hPa, 250 hPa, 100 hPa)
3133

3234
More to come!

docs/Users_Guide/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ UCAR/UCP/COMET and UCAR/NCAR/RAL. Available at: https://github.com/NCAR/wrfcloud
4242
administration
4343
user_interface
4444
graphics
45+
faqs
4546
refs
4647

4748
.. only:: html

0 commit comments

Comments
 (0)