Skip to content

Commit cf09271

Browse files
dhanur-sharmadhanur-sharma
authored
Merge pull request #1220 from NASA-IMPACT/1001-tests-for-critical-functionalities
Tests for critical functionalities
2 parents b976bac + a0a1171 commit cf09271

File tree

4 files changed

+206
-5
lines changed

4 files changed

+206
-5
lines changed

‎.github/workflows/run_full_test_suite.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,5 +33,10 @@ jobs:
3333
DJANGO_ENV: test
3434
run: docker-compose -f local.yml run --rm django bash ./init.sh
3535

36+
- name: Generate Coverage Report
37+
env:
38+
DJANGO_ENV: test
39+
run: docker-compose -f local.yml run --rm django bash -c "coverage report"
40+
3641
- name: Cleanup
3742
run: docker-compose -f local.yml down --volumes

‎docs/architecture-decisions/testing_strategy.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
## Overview
2+
As of early 2025, we have only recently been writing tests for new features, and have about 250 tests in total, mostly centered around the EJ portal, the reindexing process, and pattern applications.
3+
4+
Although this covers much of the core system logic, there still remain a number of untested logical areas such as the config file generation, core project settings, frontend features, etc.
5+
6+
This document outlines a testing strategy for the project, which will guide us towards adding tests in the most critical areas first, followed by a plan to fully cover the remaining areas.
7+
8+
## Current Coverage
9+
Using the coverage library, the following report was generated:
10+
Name | Stmts | Miss | Cover | Missing
11+
----------|--------------------------------------------------------------------------------------------------|-------|--------|--------
12+
config/__init__.py | 2 | 0 | 100% |
13+
config/celery_app.py | 6 | 0 | 100% |
14+
config/settings/__init__.py | 0 | 0 | 100% |
15+
config/settings/base.py | 94 | 0 | 100% |
16+
config/settings/local.py | 20 | 20 | 0% | 1-65
17+
config/settings/production.py | 48 | 48 | 0% | 1-162
18+
config/urls.py | 14 | 4 | 71% | 26-47
19+
config/wsgi.py | 8 | 8 | 0% | 17-36
20+
config_generation/__init__.py | 0 | 0 | 100% |
21+
config_generation/api.py | 34 | 34 | 0% | 1-88
22+
config_generation/config_example.py | 15 | 15 | 0% | 1-69
23+
config_generation/db_to_xml.py | 203 | 133 | 34% | 45, 47, 50, 96, 119-125, 129-136, 142-149, 197-200, 206-214, 225-230, 242-271, 274-278, 285-292, 303-308, 311, 317, 326-332, 342-349, 361-368, 371-374, 377-378, 382-390, 393-399, 402-412, 415-429
24+
config_generation/db_to_xml_file_based.py | 52 | 52 | 0% | 4-119
25+
config_generation/delete_config_folders.py | 24 | 24 | 0% | 9-50
26+
config_generation/delete_server_content.py | 12 | 12 | 0% | 3-25
27+
config_generation/delete_webapp_collections.py | 5 | 5 | 0% | 6-12
28+
config_generation/export_collections.py | 36 | 36 | 0% | 1-73
29+
config_generation/export_whole_index.py | 28 | 28 | 0% | 1-58
30+
config_generation/generate_collection_list.py | 29 | 29 | 0% | 8-69
31+
config_generation/generate_commands.py | 41 | 41 | 0% | 6-87
32+
config_generation/generate_emac_indexer.py | 24 | 24 | 0% | 1-81
33+
config_generation/generate_jobs.py | 42 | 42 | 0% | 8-100
34+
config_generation/generate_scrapers.py | 15 | 15 | 0% | 2-54
35+
config_generation/minimum_api.py | 33 | 33 | 0% | 1-81
36+
config_generation/preprocess_sources.py | 25 | 25 | 0% | 1-50
37+
config_generation/sources_to_scrape.py | 28 | 28 | 0% | 2-1631
38+
docs/__init__.py | 0 | 0 | 100% |
39+
docs/conf.py | 17 | 17 | 0% | 13-62
40+
environmental_justice/__init__.py | 0 | 0 | 100% |
41+
environmental_justice/admin.py | 5 | 0 | 100% |
42+
environmental_justice/apps.py | 4 | 0 | 100% |
43+
environmental_justice/models.py | 29 | 1 | 97% | 44
44+
environmental_justice/serializers.py | 6 | 0 | 100% |
45+
environmental_justice/views.py | 23 | 0 | 100% |
46+
feedback/__init__.py | 0 | 0 | 100% |
47+
feedback/admin.py | 14 | 0 | 100% |
48+
feedback/apps.py | 4 | 0 | 100% |
49+
feedback/models.py | 42 | 15 | 64% | 20-29, 35-44, 61-63
50+
feedback/serializers.py | 10 | 0 | 100% |
51+
feedback/urls.py | 4 | 0 | 100% |
52+
feedback/views.py | 9 | 0 | 100% |
53+
manage.py | 16 | 16 | 0% | 2-31
54+
merge_production_dotenvs_in_dotenv.py | 15 | 1 | 93% | 26
55+
scripts/ej/cmr_processing.py | 241 | 5 | 98% | 160, 186-188, 397, 410
56+
scripts/ej/config.py | 6 | 0 | 100% |
57+
scripts/ej/test_cmr_processing.py | 225 | 1 | 99% | 610
58+
scripts/ej/test_threshold_processing.py | 97 | 1 | 99% | 209
59+
scripts/ej/threshold_processing.py | 20 | 0 | 100% |
60+
sde_collections/__init__.py | 0 | 0 | 100% |
61+
sde_collections/admin.py | 212 | 72 | 66% | 22-24, 29, 34, 40-60, 65-81, 86-89, 98-101, 110-112, 120-134, 143, 148, 153, 158, 163, 168, 173, 178-189, 196-197, 260, 265, 270, 275, 302-303, 308-309, 314-316, 345-372, 478-480
62+
sde_collections/apps.py | 4 | 0 | 100% |
63+
sde_collections/forms.py | 15 | 0 | 100% |
64+
sde_collections/management/commands/database_backup.py | 62 | 1 | 98% | 68
65+
sde_collections/management/commands/database_restore.py | 83 | 8 | 90% | 34, 36, 87-89, 142-145
66+
sde_collections/models/__init__.py | 0 | 0 | 100% |
67+
sde_collections/models/candidate_url.py | 89 | 16 | 82% | 124, 128-134, 138-142, 145, 176-177
68+
sde_collections/models/collection.py | 414 | 144 | 65% | 241, 269, 277-287, 291-301, 305-315, 319-344, 348-357, 361, 365, 369-376, 380-387, 394, 403-406, 419, 436-439, 449-470, 478, 482-515, 519, 523, 527, 531-532, 536, 540-546, 550-553, 558-567, 575-617, 640, 679, 689, 703, 707-732, 765, 769-777, 785
69+
sde_collections/models/collection_choice_fields.py | 138 | 20 | 86% | 14-17, 36-39, 56-59, 74-77, 168-171
70+
sde_collections/models/delta_patterns.py | 313 | 33 | 89% | 119, 123, 139, 226-227, 263, 267, 291, 382-389, 439-449, 498, 503-506, 592, 627-641
71+
sde_collections/models/delta_url.py | 81 | 19 | 77% | 117-125, 129-135, 139-143, 146
72+
sde_collections/models/pattern.py | 145 | 79 | 46% | 40-48, 56-63, 66, 69, 73-74, 78-79, 87, 94-96, 105, 117-119, 128, 139-151, 163-205, 208-212, 215-216, 230-233, 243, 257-260, 268
73+
sde_collections/serializers.py | 191 | 47 | 75% | 80-81, 84-85, 88-89, 92-93, 129-130, 133-134, 137-138, 141-142, 197, 201, 211-214, 244-247, 257-260, 271, 274, 307-315, 335-343, 358-366
74+
sde_collections/sinequa_api.py | 102 | 3 | 97% | 65, 255, 289
75+
sde_collections/tasks.py | 119 | 67 | 44% | 25-67, 72-108, 113-117, 122-125, 130-148, 153-155, 215-216
76+
sde_collections/urls.py | 17 | 0 | 100% |
77+
sde_collections/utils/__init__.py | 0 | 0 | 100% |
78+
sde_collections/utils/bulk_github_push.py | 8 | 8 | 0% | 7-22
79+
sde_collections/utils/generate_deployment_message.py | 8 | 8 | 0% | 1-24
80+
sde_collections/utils/github_helper.py | 115 | 93 | 19% | 12-18, 30-42, 49-52, 60-68, 81-96, 104-110, 119-123, 127-129, 132-142, 145-152, 155-172, 175, 178-185, 189-192, 196-224, 227
81+
sde_collections/utils/health_check.py | 123 | 106 | 14% | 33-46, 51-57, 61-98, 102-143, 155-165, 172-187, 191-273
82+
sde_collections/utils/paired_field_descriptor.py | 33 | 2 | 94% | 35, 52
83+
sde_collections/utils/slack_utils.py | 19 | 4 | 79% | 57-58, 66-67
84+
sde_collections/utils/title_resolver.py | 90 | 5 | 94% | 64, 75, 83, 85, 92
85+
sde_collections/views.py | 368 | 229 | 38% | 70, 82-89, 102-141, 144-187, 194, 208-212, 215-223, 226-237, 246, 249-251, 256-265, 273-277, 280-306, 309-315, 323-327, 330-336, 339-345, 353-355, 358-368, 410, 413-422, 430, 433-442, 450, 458, 461-475, 483, 486-490, 505-511, 523-530, 538-566, 577-583, 586-607, 610-613, 628-634
86+
sde_indexing_helper/__init__.py | 2 | 0 | 100% |
87+
sde_indexing_helper/conftest.py | 9 | 0 | 100% |
88+
sde_indexing_helper/contrib/__init__.py | 0 | 0 | 100% |
89+
sde_indexing_helper/contrib/sites/__init__.py | 0 | 0 | 100% |
90+
sde_indexing_helper/users/__init__.py | 0 | 0 | 100% |
91+
sde_indexing_helper/users/adapters.py | 11 | 11 | 0% | 1-16
92+
sde_indexing_helper/users/admin.py | 13 | 0 | 100% |
93+
sde_indexing_helper/users/apps.py | 10 | 0 | 100% |
94+
sde_indexing_helper/users/context_processors.py | 3 | 0 | 100% |
95+
sde_indexing_helper/users/forms.py | 15 | 0 | 100% |
96+
sde_indexing_helper/users/models.py | 10 | 0 | 100% |
97+
sde_indexing_helper/users/tasks.py | 6 | 0 | 100% |
98+
sde_indexing_helper/users/urls.py | 4 | 0 | 100% |
99+
sde_indexing_helper/users/views.py | 27 | 0 | 100% |
100+
sde_indexing_helper/utils/__init__.py | 0 | 0 | 100% |
101+
sde_indexing_helper/utils/exceptions.py | 7 | 0 | 100% |
102+
sde_indexing_helper/utils/storages.py | 7 | 7 | 0% | 1-11
103+
tests/test_merge_production_dotenvs_in_dotenv.py | 13 | 0 | 100 |%
104+
105+
## Critical Areas
106+
### Config Generation
107+
- config_generation/db_to_xml.py
108+
- update_or_add_element_value()
109+
- _update_config_xml()
110+
- convert_template_to_scraper()
111+
- add_document_type()
112+
- add_url_exclude()
113+
- add_title_mapping()
114+
- add_job_list_item()
115+
- get_tag_value()
116+
- fetch_treeroot()
117+
- fetch_document_type()
118+
- config_generation/generate_jobs.py
119+
- make_all_parallel_jobs()
120+
121+
### Models
122+
- environmental_justice/models.py
123+
- sde_collections/models/collection.py
124+
- clear_delta_urls()
125+
- clear_dump_urls()
126+
- refresh_url_lists_for_all_patterns ()
127+
- migrate_dump_to_delta ()
128+
- create_or_update_delta_url
129+
- promote_to_curate
130+
- add_to_public_query()
131+
- create_scraper_config()
132+
- create_indexer_config()
133+
- create_plugin_config()
134+
- _write_to_github()
135+
- update_config_xml()
136+
- apply_all_patterns()
137+
- create_configs_on_status_change()
138+
- sde_collections/models/collection_choice_fields.py
139+
- sde_collections/models/delta_patterns.py
140+
- sde_collections/models/delta_url.py
141+
- sde_collections/models/pattern.py
142+
- sde_indexing_helper/users/models.py
143+
144+
### Views
145+
- environmental_justice/views.py
146+
- sde_collections/views.py
147+
- sde_indexing_helper/users/views.py
148+
149+
### Serializers and APIs
150+
- environmental_justice/serializers.py
151+
- sde_collections/serializers.py
152+
153+
### Admin Interface
154+
- environmental_justice/admin.py
155+
- sde_collections/admin.py
156+
- fetch_full_text_lrm_dev_action()
157+
- fetch_full_text_xli_action()
158+
- sde_indexing_helper/users/admin.py
159+
160+
### Utilities and Helpers
161+
- sde_collections/utils/github_helper.py
162+
- sde_collections/utils/health_check.py
163+
- sde_collections/utils/title_resolver.py
164+
- sde_collections/utils/github_helper.py
165+
- fetch_metadata()
166+
- _get_contents_from_path()
167+
168+
### Task Automation and Background Jobs
169+
- sde_collections/tasks.py
170+
171+
### Key Operational Pipelines in the Repository
172+
The selection of critical areas for testing is guided by the following pipelines of the repository:
173+
1. Sinequa config files are generated
174+
2. COSMOS imports data from LRM Dev
175+
3. Imported data is processed
176+
4. Curators update URL metadata
177+
5. Sinequa reads results from the COSMOS APIs
178+
179+
### Critical Areas Lacking Tests
180+
- **Config Generation**: Config generation files are under-tested. Develop unit tests for all critical functions in the config_generation files.
181+
- **Project Settings**: Environment-specific configurations (`local.py`, `production.py`) have no tests.
182+
- **Frontend Features**: Currently, there are no tests covering frontend logic and interactions.
183+
- **Utilities and Helpers**: Essential utility modules like github_helper.py and health_check.py lack tests
184+

‎init.sh‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/bin/bash
2-
echo "Running all test cases across the project..."
2+
echo "Running all test cases across the project with coverage analysis..."
33

44
# Initialize a failure counter
55
failure_count=0
@@ -10,10 +10,20 @@ excluded_dirs="document_classifier functional_tests"
1010
# Find all test files except those in excluded directories
1111
test_files=$(find . -type f -name "test_*.py" | grep -Ev "$(echo $excluded_dirs | sed 's/ /|/g')")
1212

13-
# Run each test file
13+
coverage erase # Clear any existing coverage data
14+
15+
# Setup .coveragerc configuration to include all Python files
16+
echo "[run]
17+
source = .
18+
include = */*.py
19+
20+
[report]
21+
show_missing = True" > .coveragerc
22+
23+
# Run each test file with coverage (without generating report yet)
1424
for test_file in $test_files; do
1525
echo "Running $test_file..."
16-
pytest "$test_file"
26+
coverage run --append -m pytest "$test_file" # Collect coverage data
1727

1828
# Check the exit status of pytest
1929
if [ $? -ne 0 ]; then
@@ -22,10 +32,11 @@ for test_file in $test_files; do
2232
fi
2333
done
2434

25-
# Report the results
35+
# Report the results without generating the coverage report
2636
if [ $failure_count -ne 0 ]; then
27-
echo "$failure_count test(s) failed."
37+
echo "$failure_count test(s) failed. Refer to the terminal output for details."
2838
exit 1
2939
else
3040
echo "All tests passed successfully!"
41+
echo "Coverage data collected. Coverage report will be generated separately."
3142
fi

‎requirements/local.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ pytest==8.0.0 # https://github.com/pytest-dev/pytest
1313
pytest-sugar==1.0.0 # https://github.com/Frozenball/pytest-sugar
1414
types-requests # maybe instead, we should add `mypy --install-types` to the dockerfile?
1515
types-xmltodict
16+
coverage
1617

1718
# Documentation
1819
# ------------------------------------------------------------------------------

0 commit comments

Comments
 (0)