Merge branch 'dev' into 1030-resolve-0-value-document-type-in-nasa_science

Author: bishwaspraveen

CHANGELOG.md

@@ -12,6 +12,22 @@ For each PR made, an entry should be added to this changelog. It should contain
   - etc.
 
 ## Changelog
+- 2889-serialize-the-tdamm-tags
+  - Description: Have TDAMM serialzed in a specific way and exposed via the Curated URLs API to be consumed into SDE Test/Prod
+  - Changes:
+    - Changed `get_tdamm_tag` method in the `CuratedURLAPISerializer` to process the TDAMM tags and pass them to the API endpoint
+
+- 960-notifications-add-a-dropdown-with-options-on-the-feedback-form
+  - Description: Generate an API endpoint and publish all the dropdown options necessary as a list for LRM to consume it.
+  - Changes:
+    - Created a new model `FeedbackFormDropdown`
+    - Added the migration file
+    - Added the `dropdown_option` field to the `Feedback` model
+    - Updated the slack notification structure by adding the dropdown option text
+    - Created a new serializer called `FeedbackFormDropdownSerializer`
+    - Added a new API endpoint `feedback-form-dropdown-options-api/` where the list is going to be accesible
+    - Added a list view called `FeedbackFormDropdownListView`
+    - Added tests
 
 - 1217-add-data-validation-to-the-feedback-form-api-to-restrict-html-content
   - Description: The feedback form API does not currently have any form of data validation on the backend which makes it easy for the user with the endpoint to send in data with html tags. We need to have a validation scheme on the backend to protect this from happening.

compose/local/django/Dockerfile

@@ -52,6 +52,7 @@ RUN apt-get update && apt-get install --no-install-recommends -y \
   && wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
   && apt-get update \
   && apt-get install -y postgresql-15 postgresql-client-15 \
+  && apt-get install -y chromium chromium-driver \
   # cleaning up unused files
   && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
   && rm -rf /var/lib/apt/lists/*

docs/test/frontend_testing_methodologies.md

@@ -0,0 +1,283 @@
+# Frontend Testing Methodologies for Django Projects
+## Overview
+
+This document outlines testing methodologies for Django projects with HTML forms and JavaScript enhancements, focusing on Python-based testing solutions. While going through the codebase, I can see it is primarily a JavaScript-heavy frontend that uses plain HTML forms enhanced with JavaScript/jQuery rather than server-rendered Django forms. Django forms are being used only in the admin panel of the project.
+
+## Primary Testing Tools
+
+### 1. Selenium with Python (Chosen)
+
+#### Capabilities
+- Full browser automation
+- JavaScript execution support
+- Real DOM interaction
+- Cross-browser testing
+- Modal dialog handling
+- AJAX request testing
+- File upload testing
+- DataTables interaction
+
+#### Implementation
+```python
+from selenium import webdriver
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.support import expected_conditions as EC
+
+class TestCollectionDetail:
+    def setup_method(self):
+        self.driver = webdriver.Chrome()
+        self.wait = WebDriverWait(self.driver, 10)
+
+    def test_title_change_modal(self):
+        # Example test for title change modal
+        self.driver.get("/collections/1/")
+        title_button = self.wait.until(
+            EC.element_to_be_clickable((By.ID, "change-title-btn"))
+        )
+        title_button.click()
+        
+        modal = self.wait.until(
+            EC.visibility_of_element_located((By.ID, "title-modal"))
+        )
+        
+        form = modal.find_element(By.TAG_NAME, "form")
+        input_field = form.find_element(By.NAME, "title")
+        input_field.send_keys("New Title")
+        form.submit()
+        
+        # Wait for AJAX completion
+        self.wait.until(
+            EC.text_to_be_present_in_element((By.ID, "collection-title"), "New Title")
+        )
+```
+
+#### Pros
+- Complete end-to-end testing
+- Real browser interaction
+- JavaScript support
+- Comprehensive API
+- Strong community support
+
+#### Drawbacks
+- Slower execution
+- Browser dependencies
+- More complex setup
+- Can be flaky with timing issues
+
+### 2. pytest-django with django-test-client
+
+#### Capabilities
+- Form submission testing
+- Response validation
+- Header verification
+- Status code checking
+- Session handling
+- Template rendering testing
+
+#### Implementation
+```python
+import pytest
+from django.urls import reverse
+
+@pytest.mark.django_db
+class TestCollectionForms:
+    def test_collection_create(self, client):
+        url = reverse('collection_create')
+        data = {
+            'title': 'Test Collection',
+            'division': 'division1',
+            'workflow_status': 'active'
+        }
+        response = client.post(url, data)
+        assert response.status_code == 302  # Redirect after success
+        
+        # Verify creation
+        response = client.get(reverse('collection_detail', kwargs={'pk': 1}))
+        assert 'Test Collection' in response.content.decode()
+```
+
+#### Pros
+- Fast execution
+- No browser dependency
+- Simpler setup
+- Integrated with Django
+
+#### Drawbacks
+- **No JavaScript support (Dealbreaker)**
+- Limited DOM interaction
+- Can't test real user interactions
+
+### 3. Playwright for Python
+
+#### Capabilities
+- Modern browser automation
+- Async/await support
+- Network interception
+- Mobile device emulation
+- Automatic waiting
+- Screenshot and video capture
+
+#### Implementation
+```python
+from playwright.sync_api import sync_playwright
+
+def test_modal_form_submission():
+    with sync_playwright() as p:
+        browser = p.chromium.launch()
+        page = browser.new_page()
+        
+        page.goto("/collections/")
+        
+        # Click button to open modal
+        page.click("#add-collection-btn")
+        
+        # Fill form in modal
+        page.fill("#title-input", "New Collection")
+        page.fill("#division-input", "Division A")
+        
+        # Submit form
+        page.click("#submit-btn")
+        
+        # Wait for success message
+        success_message = page.wait_for_selector(".toast-success")
+        assert "Collection created" in success_message.text_content()
+        
+        browser.close()
+```
+
+#### Pros
+- Modern API design
+- Better stability than Selenium
+- Built-in async support
+- Powerful debugging tools
+
+#### Drawbacks
+- **Newer tool, smaller community (Dealbreaker)**
+- Additional system dependencies
+- Learning curve for async features
+
+
+### 4. Beautiful Soup with Requests
+A combination for testing HTML structure and content.
+
+**Capabilities:**
+- HTML parsing and validation
+- Content extraction
+- Structure verification
+- Link checking
+- Form field validation
+- Template testing
+
+**Pros:**
+- Lightweight solution
+- Flexible HTML parsing
+- No browser dependency
+- Fast execution
+- Simple API
+- Low resource usage
+
+**Drawbacks:**
+- **No JavaScript support (Dealbreaker)**
+- Limited interaction testing
+- No visual testing
+- Basic functionality only
+- No real browser simulation
+
+## Feature Comparison Table
+
+| Feature                    | Selenium | Django Test Client | Playwright | Beautiful Soup |
+|---------------------------|----------|-------------------|------------|----------------|
+| JavaScript Support        | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Setup Complexity          | 🟡 Medium   | 🟢 Low            | 🟡 Medium   | 🟢 Low         |
+| Execution Speed           | 🔴 Slow   | 🟢 Fast           | 🟡 Medium   | 🟢 Fast        |
+| Modal Testing             | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| AJAX Testing              | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Cross-browser Testing     | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Real User Interaction     | ✅ Yes    | ❌ No             | ✅ Yes      | ❌ No          |
+| Documentation Quality    | ✅ Excellent| ✅ Good          | ✅ Good     | ✅ Good        |
+| Community Support        | ✅ Large   | ✅ Large          | 🟡 Growing  | ✅ Large       |
+
+## Testing Strategy Recommendations
+
+1. **Primary Testing Tool**: Selenium with Python
+   - Best suited for your JavaScript-heavy interface
+   - Handles modals and AJAX naturally
+   - Extensive documentation and community support
+
+2. **Test Coverage Areas**:
+   - Modal form interactions
+   - AJAX submissions
+   - DataTables functionality
+   - Form validation
+   - Success/error messages
+   - URL routing
+   - DOM updates
+
+## Implementation Steps
+
+1. Add testing dependencies to requirements file `requirements/local.txt`:
+```text
+# Testing Dependencies
+selenium>=4.15.2
+pytest-xdist>=3.3.1
+pytest-cov>=4.1.0
+```
+
+2. Update Dockerfile `compose/local/django/Dockerfile` to install Chrome and ChromeDriver:
+```dockerfile
+# Install Chrome and ChromeDriver for Selenium tests
+RUN apt-get update && apt-get install -y \
+    chromium \
+    chromium-driver \
+    && rm -rf /var/lib/apt/lists/*
+```
+
+3. Rebuild Docker container to apply changes:
+```bash
+docker-compose -f local.yml build django
+```
+
+4. Create test directory structure:
+```bash
+mkdir -p tests/frontend
+touch tests/frontend/__init__.py
+touch tests/frontend/base.py
+touch tests/frontend/test_setup.py
+```
+
+5. Create base test classes:
+```python
+import pytest
+from selenium import webdriver
+
+class BaseUITest:
+    @pytest.fixture(autouse=True)
+    def setup_class(self):
+        self.driver = webdriver.Chrome()
+        yield
+        self.driver.quit()
+
+    def login(self):
+        # Common login logic
+        pass
+```
+
+6. Organize tests by feature:
+```python
+class TestCollectionManagement(BaseUITest):
+    def test_create_collection(self):
+        pass
+
+    def test_edit_collection(self):
+        pass
+
+class TestURLPatterns(BaseUITest):
+    def test_add_include_pattern(self):
+        pass
+```
+
+7. Run tests:
+```bash
+docker-compose -f local.yml run --rm django pytest tests/frontend/test_setup.py -v
+```

feedback/migrations/0005_feedbackformdropdown_feedback_dropdown_option.py

@@ -0,0 +1,37 @@
+# Generated by Django 4.2.9 on 2025-01-29 19:27
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("feedback", "0004_contentcurationrequest_created_at_and_more"),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name="FeedbackFormDropdown",
+            fields=[
+                ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+                ("name", models.CharField(max_length=200)),
+                ("display_order", models.PositiveIntegerField(default=1)),
+            ],
+            options={
+                "verbose_name": "Dropdowm Option",
+                "verbose_name_plural": "Dropdown Options",
+                "ordering": ["display_order", "name"],
+            },
+        ),
+        migrations.AddField(
+            model_name="feedback",
+            name="dropdown_option",
+            field=models.ForeignKey(
+                null=True,
+                on_delete=django.db.models.deletion.SET_NULL,
+                related_name="feedback",
+                to="feedback.feedbackformdropdown",
+            ),
+        ),
+    ]