Merge pull request #1226 from NASA-IMPACT/1195-implement-unit-test-for-forms-on-the-frontend

Implement unit test for forms on the frontend

Author: dhanur-sharma

compose/local/django/Dockerfile

@@ -52,6 +52,7 @@ RUN apt-get update && apt-get install --no-install-recommends -y \
   && wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
   && apt-get update \
   && apt-get install -y postgresql-15 postgresql-client-15 \
+  && apt-get install -y chromium chromium-driver \
   # cleaning up unused files
   && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
   && rm -rf /var/lib/apt/lists/*

docs/test/frontend_testing_methodologies.md

@@ -0,0 +1,283 @@
+# Frontend Testing Methodologies for Django Projects
+## Overview
+
+This document outlines testing methodologies for Django projects with HTML forms and JavaScript enhancements, focusing on Python-based testing solutions. While going through the codebase, I can see it is primarily a JavaScript-heavy frontend that uses plain HTML forms enhanced with JavaScript/jQuery rather than server-rendered Django forms. Django forms are being used only in the admin panel of the project.
+
+## Primary Testing Tools
+
+### 1. Selenium with Python (Chosen)
+
+#### Capabilities
+- Full browser automation
+- JavaScript execution support
+- Real DOM interaction
+- Cross-browser testing
+- Modal dialog handling
+- AJAX request testing
+- File upload testing
+- DataTables interaction
+
+#### Implementation
+```python
+from selenium import webdriver
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.support import expected_conditions as EC
+
+class TestCollectionDetail:
+    def setup_method(self):
+        self.driver = webdriver.Chrome()
+        self.wait = WebDriverWait(self.driver, 10)
+
+    def test_title_change_modal(self):
+        # Example test for title change modal
+        self.driver.get("/collections/1/")
+        title_button = self.wait.until(
+            EC.element_to_be_clickable((By.ID, "change-title-btn"))
+        )
+        title_button.click()
+        
+        modal = self.wait.until(
+            EC.visibility_of_element_located((By.ID, "title-modal"))
+        )
+        
+        form = modal.find_element(By.TAG_NAME, "form")
+        input_field = form.find_element(By.NAME, "title")
+        input_field.send_keys("New Title")
+        form.submit()
+        
+        # Wait for AJAX completion
+        self.wait.until(
+            EC.text_to_be_present_in_element((By.ID, "collection-title"), "New Title")
+        )
+```
+
+#### Pros
+- Complete end-to-end testing
+- Real browser interaction
+- JavaScript support
+- Comprehensive API
+- Strong community support
+
+#### Drawbacks
+- Slower execution
+- Browser dependencies
+- More complex setup
+- Can be flaky with timing issues
+
+### 2. pytest-django with django-test-client
+
+#### Capabilities
+- Form submission testing
+- Response validation
+- Header verification
+- Status code checking
+- Session handling
+- Template rendering testing
+
+#### Implementation
+```python
+import pytest
+from django.urls import reverse
+
+@pytest.mark.django_db
+class TestCollectionForms:
+    def test_collection_create(self, client):
+        url = reverse('collection_create')
+        data = {
+            'title': 'Test Collection',
+            'division': 'division1',
+            'workflow_status': 'active'
+        }
+        response = client.post(url, data)
+        assert response.status_code == 302  # Redirect after success
+        
+        # Verify creation
+        response = client.get(reverse('collection_detail', kwargs={'pk': 1}))
+        assert 'Test Collection' in response.content.decode()
+```
+
+#### Pros
+- Fast execution
+- No browser dependency
+- Simpler setup
+- Integrated with Django
+
+#### Drawbacks
+- **No JavaScript support (Dealbreaker)**
+- Limited DOM interaction
+- Can't test real user interactions
+
+### 3. Playwright for Python
+
+#### Capabilities
+- Modern browser automation
+- Async/await support
+- Network interception
+- Mobile device emulation
+- Automatic waiting
+- Screenshot and video capture
+
+#### Implementation
+```python
+from playwright.sync_api import sync_playwright
+
+def test_modal_form_submission():
+    with sync_playwright() as p:
+        browser = p.chromium.launch()
+        page = browser.new_page()
+        
+        page.goto("/collections/")
+        
+        # Click button to open modal
+        page.click("#add-collection-btn")
+        
+        # Fill form in modal
+        page.fill("#title-input", "New Collection")
+        page.fill("#division-input", "Division A")
+        
+        # Submit form
+        page.click("#submit-btn")
+        
+        # Wait for success message
+        success_message = page.wait_for_selector(".toast-success")
+        assert "Collection created" in success_message.text_content()
+        
+        browser.close()
+```
+
+#### Pros
+- Modern API design
+- Better stability than Selenium
+- Built-in async support
+- Powerful debugging tools
+
+#### Drawbacks
+- **Newer tool, smaller community (Dealbreaker)**
+- Additional system dependencies
+- Learning curve for async features
+
+
+### 4. Beautiful Soup with Requests
+A combination for testing HTML structure and content.
+
+**Capabilities:**
+- HTML parsing and validation
+- Content extraction
+- Structure verification
+- Link checking
+- Form field validation
+- Template testing
+
+**Pros:**
+- Lightweight solution
+- Flexible HTML parsing
+- No browser dependency
+- Fast execution
+- Simple API
+- Low resource usage
+
+**Drawbacks:**
+- **No JavaScript support (Dealbreaker)**
+- Limited interaction testing
+- No visual testing
+- Basic functionality only
+- No real browser simulation
+
+## Feature Comparison Table
+
+| Feature                    | Selenium | Django Test Client | Playwright | Beautiful Soup |
+|---------------------------|----------|-------------------|------------|----------------|
+| JavaScript Support        | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Setup Complexity          | 🟡 Medium   | 🟢 Low            | 🟡 Medium   | 🟢 Low         |
+| Execution Speed           | 🔴 Slow   | 🟢 Fast           | 🟡 Medium   | 🟢 Fast        |
+| Modal Testing             | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| AJAX Testing              | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Cross-browser Testing     | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Real User Interaction     | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Documentation Quality    | ✅ Excellent| ✅ Good          | ✅ Good     | ✅ Good        |
+| Community Support        | ✅ Large   | ✅ Large          | 🟡 Growing  | ✅ Large       |
+
+## Testing Strategy Recommendations
+
+1. **Primary Testing Tool**: Selenium with Python
+   - Best suited for your JavaScript-heavy interface
+   - Handles modals and AJAX naturally
+   - Extensive documentation and community support
+
+2. **Test Coverage Areas**:
+   - Modal form interactions
+   - AJAX submissions
+   - DataTables functionality
+   - Form validation
+   - Success/error messages
+   - URL routing
+   - DOM updates
+
+## Implementation Steps
+
+1. Add testing dependencies to requirements file `requirements/local.txt`:
+```text
+# Testing Dependencies
+selenium>=4.15.2
+pytest-xdist>=3.3.1
+pytest-cov>=4.1.0
+```
+
+2. Update Dockerfile `compose/local/django/Dockerfile` to install Chrome and ChromeDriver:
+```dockerfile
+# Install Chrome and ChromeDriver for Selenium tests
+RUN apt-get update && apt-get install -y \
+    chromium \
+    chromium-driver \
+    && rm -rf /var/lib/apt/lists/*
+```
+
+3. Rebuild Docker container to apply changes:
+```bash
+docker-compose -f local.yml build django
+```
+
+4. Create test directory structure:
+```bash
+mkdir -p tests/frontend
+touch tests/frontend/__init__.py
+touch tests/frontend/base.py
+touch tests/frontend/test_setup.py
+```
+
+5. Create base test classes:
+```python
+import pytest
+from selenium import webdriver
+
+class BaseUITest:
+    @pytest.fixture(autouse=True)
+    def setup_class(self):
+        self.driver = webdriver.Chrome()
+        yield
+        self.driver.quit()
+
+    def login(self):
+        # Common login logic
+        pass
+```
+
+6. Organize tests by feature:
+```python
+class TestCollectionManagement(BaseUITest):
+    def test_create_collection(self):
+        pass
+
+    def test_edit_collection(self):
+        pass
+
+class TestURLPatterns(BaseUITest):
+    def test_add_include_pattern(self):
+        pass
+```
+
+7. Run tests:
+```bash
+docker-compose -f local.yml run --rm django pytest tests/frontend/test_setup.py -v
+```

requirements/local.txt

@@ -13,7 +13,10 @@ pytest==8.0.0  # https://github.com/pytest-dev/pytest
 pytest-sugar==1.0.0  # https://github.com/Frozenball/pytest-sugar
 types-requests # maybe instead, we should add `mypy --install-types` to the dockerfile?
 types-xmltodict
-coverage
+pytest-xdist>=3.3.1
+pytest-cov>=4.1.0
+selenium>=4.15.2    # Selenium (Frontend Testing)
+coverage==7.4.1
 
 # Documentation
 # ------------------------------------------------------------------------------

sde_collections/tests/frontend/__init__.py

@@ -0,0 +1,50 @@
+import shutil
+
+import pytest
+from django.contrib.staticfiles.testing import StaticLiveServerTestCase
+from selenium import webdriver
+from selenium.webdriver.chrome.options import Options
+from selenium.webdriver.chrome.service import Service
+from selenium.webdriver.support.ui import WebDriverWait
+
+from .mixins import AuthenticationMixin
+
+
+class BaseTestCase(StaticLiveServerTestCase, AuthenticationMixin):
+    """Base class for all frontend tests using Selenium."""
+
+    @classmethod
+    def setUpClass(cls):
+        super().setUpClass()
+
+        # Verify ChromeDriver and Chromium are available
+        chromedriver_path = shutil.which("chromedriver")
+        chromium_path = shutil.which("chromium")
+
+        if not chromedriver_path:
+            pytest.fail("ChromeDriver not found. Please ensure chromium-driver is installed.")
+        if not chromium_path:
+            pytest.fail("Chromium not found. Please ensure chromium is installed.")
+
+        # Set up Chrome options
+        chrome_options = Options()
+        chrome_options.add_argument("--headless")
+        chrome_options.add_argument("--no-sandbox")
+        chrome_options.add_argument("--disable-dev-shm-usage")
+        chrome_options.binary_location = chromium_path
+
+        try:
+            service = Service(executable_path=chromedriver_path)
+            cls.driver = webdriver.Chrome(service=service, options=chrome_options)
+            cls.driver.set_window_size(1920, 1080)
+            cls.driver.implicitly_wait(10)
+            cls.wait = WebDriverWait(cls.driver, 10)
+
+        except Exception as e:
+            pytest.fail(f"Failed to initialize ChromeDriver: {str(e)}")
+
+    @classmethod
+    def tearDownClass(cls):
+        if hasattr(cls, "driver"):
+            cls.driver.quit()
+        super().tearDownClass()