|
| 1 | +<p align="center"> |
| 2 | + <img alt="NSIDC logo" src="https://nsidc.org/themes/custom/nsidc/logo.svg" width="150" /> |
| 3 | +</p> |
| 4 | + |
1 | 5 | # Data Access Tool (DAT) Backend |
2 | 6 |
|
3 | 7 | Backend services for the |
4 | 8 | [Data Access Tool UI](https://github.com/nsidc/data-access-tool-ui). |
5 | 9 |
|
6 | | -This repository provides: |
7 | | - |
8 | | -- A Flask-based API provides: |
9 | | - |
10 | | -* The`python_script.py` download script for data granules matching a user's |
11 | | - filters. |
12 | | -* The |
13 | | - [getLinks](https://github.com/nasa/earthdata-download/blob/main/docs/GET_LINKS.md) |
14 | | - service required for DAT integration with the |
15 | | - [NASA Earthdata Downloader](https://github.com/nasa/earthdata-download). |
16 | | - |
17 | | -- Docker compose configuration for the DAT backend, which includes |
18 | | - [nginx server configuration](./nginx). |
19 | | - |
20 | | -## Background |
21 | | - |
22 | | -This service was originally a part of the |
23 | | -[hermes-api](https://bitbucket.org/nsidc/hermes-api/src). It was moved to a |
24 | | -standalone service to support the decommissioning of ECS and the rest of the |
25 | | -hermes stack planned for July 2026. |
26 | | - |
27 | | -## Dev |
28 | | - |
29 | | -### Pre-commit |
30 | | - |
31 | | -This project uses [pre-commit](https://pre-commit.com/) to run pre-commit hooks |
32 | | -that check and format this project's code for stylistic consistency (e.g., using |
33 | | -`ruff` and `black`) . |
34 | | - |
35 | | -The pre-commit configuration for this project can be found in |
36 | | -`.pre-commit-config.yaml`. Configuration for specific tools (e.g., `vulture`) is |
37 | | -given in the included `pyproject.toml`. |
38 | | - |
39 | | -For more information about using `pre-commit`, please sese the |
40 | | -[Scientific Python Library Development Guide's section on pre-commit](https://learn.scientific-python.org/development/guides/gha-basic/#pre-commit). |
41 | | - |
42 | | -To install pre-commit to run checks for each commit you make: |
43 | | - |
44 | | -``` |
45 | | -$ pre-commit install |
46 | | -``` |
47 | | - |
48 | | -To manually run the pre-commit hooks without a commit: |
49 | | - |
50 | | -``` |
51 | | -$ pre-commit run --all-files |
52 | | -``` |
53 | | - |
54 | | -> [!NOTE] GitHub actions are configured to run pre-commit for all PRs and pushes |
55 | | -> to the `main` branch. See |
56 | | -> [.github/workflows/pre-commit.yml](.github/workflows/pre-commit.yml). |
57 | | -
|
58 | | -### Running tests |
59 | | - |
60 | | -Before manually running tests, setup the `EARTHDATA_USERNAME` and |
61 | | -`EARTHDATA_PASSWORD` envvars, which are necessary for integration tests. |
62 | | - |
63 | | -Next, to run all tests: |
64 | | - |
65 | | -``` |
66 | | -scripts/run_tests.sh |
67 | | -``` |
68 | | - |
69 | | -> [!NOTE] GitHub actions are configured to run unit tests that do not require |
70 | | -> Earthdata login credentials for all PRs and pushes to the `main` branch. See |
71 | | -> [.github/workflows/test.yml](.github/workflows/test.yml). |
72 | | -
|
73 | | -### Testing the EDD integration |
74 | | - |
75 | | -An example deep-link to initiate EDD downloads: |
76 | | - |
77 | | -In integration (note: for the get-links query param, the `cmr_request_params` |
78 | | -were url-encoded, and then the entire URL was url-encoded): |
| 10 | +## Level of Support |
79 | 11 |
|
80 | | -``` |
81 | | -earthdata-download://startDownload?getLinks=https%3A//integration.nsidc.org/apps/data-access-tool/api/get-links%3Fcmr_request_params%3Dprovider%253DNSIDC_CPRD%2526page_size%253D5%2526sort_key%255B%255D%253D-start_date%2526sort_key%255B%255D%253Dproducer_granule_id%2526short_name%253DATL06%2526version%253D6%2526version%253D06%2526version%253D006%2526temporal%255B%255D%253D2018-10-14T00%253A00%253A00Z%252C2025-02-25T00%253A25%253A20Z%2526bounding_box%253D-180%252C-90%252C180%252C90%2526options%255Bproducer_granule_id%255D%255Bpattern%255D%253Dtrue%2526producer_granule_id%255B%255D%253D%252AATL06_2024%252A_0804%252A_006_01.h5%252A&downloadId=atl06_06&clientId=data_access_tool&authUrl=https://integration.nsidc.org/apps/data-access-tool/api/earthdata/auth?eddRedirect=earthdata-download%3A%2F%2FauthCallback |
82 | | -``` |
| 12 | +This repository is fully supported by NSIDC. If you discover any problems or |
| 13 | +bugs, please submit an Issue. If you would like to contribute to this |
| 14 | +repository, you may fork the repository and submit a pull request. |
83 | 15 |
|
84 | | -A button needs to be added to the Data Access Tool that will issue a GET request |
85 | | -that looks like the above. |
| 16 | +See the [LICENSE](LICENSE) for details on permissions and warranties. Please |
| 17 | +contact nsidc@nsidc.org for more information. |
86 | 18 |
|
87 | | -The GET request to `earthdata-download://startDownload` should include the |
88 | | -following query parameters: |
| 19 | +## Requirements |
89 | 20 |
|
90 | | -- `getLinks`: URI for `/api/get-links/`. This URI will specify the |
91 | | - `cmr_request_params` query-parameter, which is a string representing the CMR |
92 | | - query parameters mapping to a user's selections in the DAT. |
93 | | -- `downloadId`: The dataset ID and version for the current order (e.g., ATL06 v6 |
94 | | - is `atl06_06`) |
95 | | -- `authUrl`: URI for `/api/earthdata/auth/`. EDD will use this to initiate a |
96 | | - token exchange with URS to authenticate user downloads. This URL must include |
97 | | - `eddRedirect=earthdata-download%3A%2F%2FauthCallback` as a query parameter. |
| 21 | +- [Docker](https://www.docker.com/) and |
| 22 | + [docker compose](https://docs.docker.com/compose/) |
| 23 | +- Access to NSIDC's internal Virtual Machine infrastructure. It is expected that |
| 24 | + this backend system be deployed via the |
| 25 | + [data-access-tool-vm](https://github.com/nsidc/data-access-tool-vm) project. |
98 | 26 |
|
99 | | -> [!WARNING] As of this writing, the CMR query parameters are hard-coded to |
100 | | -> always return a small subset of ATL06 v6 data. |
| 27 | +## Contributing |
101 | 28 |
|
102 | | -## Releasing |
| 29 | +Contributing documentation can be viewed by running `quarto preview docs/` |
103 | 30 |
|
104 | | -To release a new version: |
| 31 | +## Credit |
105 | 32 |
|
106 | | -- Make changes on a branch |
107 | | -- Update CHANGELOG for next release |
108 | | -- Run `bump-my-version bump {major|minor|patch}` |
109 | | -- Open a PR and have it merged to `main` after review |
110 | | -- Tag latest commit on `main` with the version, and push. This will trigger a |
111 | | - build of the `data-access-tool-api` and `data-access-tool-server` images with |
112 | | - the given version tag. |
113 | | -- Deploy the latest change with the |
114 | | - [data-access-tool-vm](https://github.com/nsidc/data-access-tool-vm). |
| 33 | +This content was developed by the National Snow and Ice Data Center with funding |
| 34 | +from multiple sources. |
0 commit comments