|
1 | 1 | # Metadata-driven data discovery |
2 | 2 |
|
3 | | -The NIAID Data Ecosystem Discovery Portal enables users to find datasets from a wide range of repositories and offers a convenient one-stop-show for discovery of data on infectious and immune-mediated diseases (IIDs). The Discovery Portal collects metadata from all included repositories on a regular basis and includes it in the searchable catalogue for anybody to explore. |
4 | | - |
| 3 | +The NIAID Data Ecosystem Discovery Portal enables users to find datasets from a |
| 4 | +wide range of repositories and offers a convenient one-stop-show for discovery |
| 5 | +of data on infectious and immune-mediated diseases (IIDs). The Discovery Portal |
| 6 | +collects metadata from all included repositories on a regular basis and includes |
| 7 | +it in the searchable catalogue for anybody to explore. |
5 | 8 |
|
6 | 9 | # Data access through repositories |
7 | 10 |
|
8 | | -The NIAID Data Ecosystem uses a federated and distributed architecture. No data is hosted by the Discovery Portal. The Discovery Portal does not provide direct access to datasets but provides links to dataset landing pages with their respective repositories. Users can use these links to explore datasets of interest and obtain access as required by the repository that hosts the data. |
9 | | - |
| 11 | +The NIAID Data Ecosystem uses a federated and distributed architecture. No data |
| 12 | +is hosted by the Discovery Portal. The Discovery Portal does not provide direct |
| 13 | +access to datasets but provides links to dataset landing pages with their |
| 14 | +respective repositories. Users can use these links to explore datasets of |
| 15 | +interest and obtain access as required by the repository that hosts the data. |
10 | 16 |
|
11 | 17 | The Discovery Portal can be used to: |
12 | 18 |
|
13 | | -- [Search across millions of datasets from numerous sources](https://data-staging.niaid.nih.gov/search/?q=&from=1&filters=%28%40type%3A%28%22Dataset%22%29%29), datasets that were previously unknown, to bring other dimensions into analyses. |
14 | | -- [Download metadata](https://data-staging.niaid.nih.gov/search/), access via [API](https://api.data.niaid.nih.gov/), or use the [metadata visualization tools](https://data-staging.niaid.nih.gov/summary/?q=&from=1&filters=) to gather new insights about what’s available. |
| 19 | +- [Search across millions of datasets from numerous sources](https://data-staging.niaid.nih.gov/search/?q=&from=1&filters=%28%40type%3A%28%22Dataset%22%29%29), |
| 20 | + datasets that were previously unknown, to bring other dimensions into |
| 21 | + analyses. |
| 22 | +- [Download metadata](https://data-staging.niaid.nih.gov/search/), access via |
| 23 | + [API](https://api.data.niaid.nih.gov/), or use the |
| 24 | + [metadata visualization tools](https://data-staging.niaid.nih.gov/summary/?q=&from=1&filters=) |
| 25 | + to gather new insights about what’s available. |
15 | 26 | - Track research across funding programs or specific scientific areas. |
16 | 27 |
|
17 | | - |
18 | 28 | # Interpret search results carefully |
19 | 29 |
|
20 | | -- The Discovery Portal does not aggregate every dataset from every source related to infectious and allergic disease. The Discovery Portal collects metadata from a defined list of [data repositories](https://data-staging.niaid.nih.gov/sources/) and standardizes them according to our [metadata schema](https://discovery.biothings.io/view/nde). The list is continuously updated. Anybody is welcome to [suggest a data repository here](https://github.com/NIAID-Data-Ecosystem/nde-crawlers/issues/new?assignees=&labels=&template=suggest-a-new-resource.md&title=%5BSOURCE%5D). |
21 | | - |
22 | | -- **Results may include irrelevant datasets.** The Discovery Portal also collects metadata from generalist repositories and search results may include datasets that are not related to infectious or immune-mediated diseases (IIDs). |
23 | | - |
24 | | -- **Metadata fields may be incomplete.** The Discovery Portal attempts to standardize metadata that is made available by data repositories. If metadata from the repositories is missing or incomplete, it will also be missing or incomplete within the Discovery Portal. NIAID is working with the community to improve the completeness, quality, and consistency of metadata, in accordance with the [FAIR Guiding Principles](https://doi.org/10.1038/sdata.2016.18). |
25 | | - |
| 30 | +- The Discovery Portal does not aggregate every dataset from every source |
| 31 | + related to infectious and allergic disease. The Discovery Portal collects |
| 32 | + metadata from a defined list of |
| 33 | + [data repositories](https://data-staging.niaid.nih.gov/sources/) and |
| 34 | + standardizes them according to our <Link |
| 35 | + isExternal |
| 36 | + href='https://discovery.biothings.io/view/nde'> |
| 37 | + metadata schema</Link> . The list is continuously updated. Anybody is welcome to <Link |
| 38 | + isExternal |
| 39 | + href='https://github.com/NIAID-Data-Ecosystem/nde-crawlers/issues/new?assignees=&labels=&template=suggest-a-new-resource.md&title=%5BSOURCE%5D'> |
| 40 | + suggest a data repository here</Link> . |
| 41 | + |
| 42 | +- **Results may include irrelevant datasets.** The Discovery Portal also |
| 43 | + collects metadata from generalist repositories and search results may include |
| 44 | + datasets that are not related to infectious or immune-mediated diseases |
| 45 | + (IIDs). |
| 46 | + |
| 47 | +- **Metadata fields may be incomplete.** The Discovery Portal attempts to |
| 48 | + standardize metadata that is made available by data repositories. If metadata |
| 49 | + from the repositories is missing or incomplete, it will also be missing or |
| 50 | + incomplete within the Discovery Portal. NIAID is working with the community to |
| 51 | + improve the completeness, quality, and consistency of metadata, in accordance |
| 52 | + with the <Link |
| 53 | + isExternal |
| 54 | + href='https://doi.org/10.1038/sdata.2016.18'> |
| 55 | + FAIR Guiding Principles</Link> . |
26 | 56 |
|
27 | 57 | # Partnership |
28 | | -The NIAID Data Ecosystem is being developed by a large number of partners. Foremost, the numerous data repositories who host the data and provide metadata. Data repositories include domain specific repositories for infectious and immune-mediated disease data, some of which are supported by NIAID, and also general repositories. The Discovery Portal is being developed by [Scripps Research](https://www.scripps.edu/) and built upon previous work in collaboration with Seven Bridges. NIAID is also working with partners across NIH, the US Government, and beyond to align the architecture and vision of the NIAID Ecosystem with similar systems elsewhere. The NIAID Data Ecosystem is in early stages of development and the partnerships will continue to expand as we continue to improve the capabilities and value of the Ecosystem to the research community. |
29 | | - |
30 | | - |
31 | | -For more information, visit [Frequently Asked Questions](https://data-staging.niaid.nih.gov/faq/). |
32 | 58 |
|
33 | | -To provide feedback or ask additional questions, contact us at [[email protected]](mailto:[email protected]). |
| 59 | +The NIAID Data Ecosystem is being developed by a large number of partners. |
| 60 | +Foremost, the numerous data repositories who host the data and provide metadata. |
| 61 | +Data repositories include domain specific repositories for infectious and |
| 62 | +immune-mediated disease data, some of which are supported by NIAID, and also |
| 63 | +general repositories. The Discovery Portal is being developed by <Link |
| 64 | +isExternal href='https://www.scripps.edu/'>Scripps Research</Link> and built upon previous work in |
| 65 | +collaboration with Seven Bridges. NIAID is also working with partners across |
| 66 | +NIH, the US Government, and beyond to align the architecture and vision of the |
| 67 | +NIAID Ecosystem with similar systems elsewhere. The NIAID Data Ecosystem is in |
| 68 | +early stages of development and the partnerships will continue to expand as we |
| 69 | +continue to improve the capabilities and value of the Ecosystem to the research |
| 70 | +community. |
| 71 | + |
| 72 | +For more information, visit |
| 73 | +[Frequently Asked Questions](https://data-staging.niaid.nih.gov/faq/). |
| 74 | + |
| 75 | +To provide feedback or ask additional questions, contact us at |
| 76 | + |
0 commit comments