Fix failing test suites in CI

[mwallschlaeger]

Problem

Multiple test suites are currently failing in the CI workflow (Issue #71 implementation).

Failing Test Suites

Based on the latest CI run:

1. Main Test Suite (test.sh)

• Status: Failing
• Error: Database permissions and test setup issues

2. API v2 Tests (test_api_v2.sh)

• Status: Failing
• Error: permission denied for table resource_executionrequest during paver setup_data

3. CSW Tests (test_csw.sh)

• Status: Unknown - needs investigation

4. Integration Tests (test_integration.sh)

• Status: Failing
• Error: ModuleNotFoundError: No module named 'geonode.tests.integration'
• Note: This script uses paver run_tests which has different test discovery than standard manage.py test

5. OAuth2 Tests (test_oauth2.sh)

• Status: Unknown - needs investigation

6. Upload Tests (test_upload.sh)

• Status: Unknown - needs investigation

Root Causes

1. Database Permissions: Fixed by granting SUPERUSER to geonode user (PR Add test documentation and CI workflow for Issue #71 #300)
2. Test Module Structure: Some test scripts reference non-existent test modules
3. Test Data Setup: paver setup_data may have timing or dependency issues
4. Environment Configuration: Test environment variables may need adjustment

Acceptance Criteria

• All test suites pass in CI
• Test failures are properly reported (workflow fails when tests fail)
• Test results are documented in CI summary
• Any test scripts with incorrect module paths are fixed or documented

Related

• PR Add test documentation and CI workflow for Issue #71 #300: Run tests on PR (Issue Task: run tests on PR #71)
• TESTING.md: Documentation for running tests locally

Notes

The CI infrastructure is now in place (PR #300). This issue focuses on fixing the actual test failures to get a green build.

[mwallschlaeger]

Walkthrough: Fix Failing Test Suites (Issue #301)

Understanding of the Issue

Issue #301 was created after PR #300 successfully set up the CI infrastructure for running tests on PRs. The CI is now working correctly (tests run, failures are reported), but multiple test suites are failing. The issue identifies 6 failing test suites:

1. Main Test Suite (test.sh) - Database permissions and setup issues
2. API v2 Tests (test_api_v2.sh) - Permission denied for table resource_executionrequest
3. CSW Tests (test_csw.sh) - Unknown status
4. Integration Tests (test_integration.sh) - ModuleNotFoundError: No module named 'geonode.tests.integration'
5. OAuth2 Tests (test_oauth2.sh) - Unknown status
6. Upload Tests (test_upload.sh) - Unknown status

Root Causes Identified

From the issue and recent work on PR #300:

• Database permissions were fixed by granting SUPERUSER to geonode user
• Some test scripts reference non-existent test modules
• paver setup_data has timing/dependency issues
• Environment configuration may need adjustment

Proposed Technical Solution

Scope Clarification

This issue is very large - fixing all 6 test suites could involve dozens of individual test failures. I propose we take a phased approach:

Phase 1 (This PR): Fix the most critical infrastructure issues that prevent tests from running at all:

1. Fix the test_integration.sh module path issue (ModuleNotFoundError)
2. Verify database permissions are working correctly
3. Document any remaining test failures for future PRs

Phase 2+ (Future PRs): Address individual test failures within each suite

Files to Change

1. Investigate and Document Test Failures

Action: Run each test suite locally to identify specific failures

docker compose -f docker-compose-test.yml up -d
docker compose -f docker-compose-test.yml exec django ./test.sh
docker compose -f docker-compose-test.yml exec django ./test_api_v2.sh  
docker compose -f docker-compose-test.yml exec django ./test_csw.sh
docker compose -f docker-compose-test.yml exec django ./test_integration.sh
docker compose -f docker-compose-test.yml exec django ./test_oauth2.sh
docker compose -f docker-compose-test.yml exec django ./test_upload.sh

2. Fix test_integration.sh Module Path Issue

File: test_integration.sh

Problem: The script tries to run geonode.tests.integration which doesn't exist

Solution Options:

• Option A: Update the script to use the correct test path (if integration tests exist elsewhere)
• Option B: Comment out or skip this test suite with documentation explaining why
• Option C: Create the missing test module structure

Decision: Need to investigate what integration tests actually exist

3. Update TESTING.md Documentation

File: TESTING.md

Action: Document known test failures and their status

Verification Plan

Automated Tests

1. Run test suites locally:

   docker compose -f docker-compose-test.yml up -d
   # Wait for services to be healthy
   docker compose -f docker-compose-test.yml exec django ./test.sh

2. Verify CI workflow:

   • Push changes to PR
   • Check GitHub Actions workflow runs
   • Verify test results are properly reported

Manual Verification

1. Check test output: Review logs for each test suite to identify specific failures
2. Verify documentation: Ensure TESTING.md accurately reflects current test status

Questions for User

Before proceeding, I need clarification on scope:

1. Scope: Should this PR attempt to fix ALL test failures, or should we:

   • Fix only infrastructure issues (module paths, permissions)
   • Document remaining failures for future PRs
   • Create separate issues for each failing test suite

2. Priority: Which test suite is most critical to fix first?

3. Integration Tests: The test_integration.sh script references geonode.tests.integration which doesn't exist. Should we:

   • Skip this test suite for now?
   • Investigate what integration tests should run?
   • Remove this test script entirely?

4. Acceptance Criteria: The issue states "All test suites pass in CI" - is this realistic for a single PR, or should we break this into multiple PRs?

Approved Approach

Per user request, this PR will:

1. ✅ Investigate and document all current test failures
2. ✅ Fix ALL test failures across all 6 test suites
3. ✅ Update TESTING.md with any remaining known issues
4. ✅ Ensure all tests pass in CI

This is a comprehensive fix that will result in a green build.

Next Steps

1. ✅ User approved comprehensive approach
2. Run all test suites locally to identify specific failures
3. Fix each failing test systematically
4. Verify all tests pass
5. Update documentation
6. Submit PR