Merge pull request #58 from alimkhann/feature/search-filters-pagination

Feature/search filters pagination

Author: alimkhann

NOTES.md

@@ -0,0 +1,37 @@
+# Prompt 1 — Core Data Model Upgrades (GeneratedRoadmap + User Repo State)
+
+Date: 2025-11-16
+
+## Changes made
+- Extended `generated_roadmaps` table/model with richer metadata columns:
+  - `primary_language`, `languages`, `topics`, `difficulty`, `star_count`, `fork_count`, `last_pushed_at`, `license`, `contributor_count`, `view_count`, `sync_count`, `rating_count`, `rating_sum`.
+- Extended per-user repo state (`user_synced_repos` table/model) with lifecycle and progress fields:
+  - `status`, `is_archived`, `progress_percent`, timestamps `created_at`, `updated_at` (kept `pinned_at`).
+- Updated Pydantic `RoadmapRepoSummary` to accept the new metadata fields (all optional to stay backward compatible). Existing API shapes are unchanged.
+
+## Prompt 2 additions (Your Repositories + statuses)
+- Added `UserRepoStateResponse` schema and backend endpoint `GET /api/v1/roadmap/user-repos` (auth) to return per-user repo states with status, archive flag, progress, and basic roadmap summary.
+- Enhanced `UserSyncedRepoStore` with `upsert_state` and `list_states` to manage statusful per-user repos.
+- Frontend `repoService` now has `listUserRepos`; `RoadmapCatalogProvider` fetches/stores `yourRepos` with Clerk auth token.
+- Sidebar section renamed to “Your repositories”; now shows status badges and includes unsynced entries (non-archived), falling back to old synced list if no user data.
+- Search page “Your repositories” section uses the new user-repo list and keeps stage counts via cached synced map.
+
+## Public Repositories Catalog (Prompt 3)
+- Backend `/api/v1/roadmap/catalog` now paginated (`items`, `page`, `page_size`, `total_count`, `total_pages`) backed by `GeneratedRoadmap` with new metadata included.
+- Roadmap service exposes `list_catalog`; repository store supports paginated listing.
+- Frontend `repoService.listCatalog(page, pageSize)` returns catalog page; `RoadmapCatalogProvider` consumes `.items` to populate synced list.
+- Search page renders a separate “Public repositories” section using the new endpoint (with static fallback when API is unconfigured). Displays stage count, language, and links to timelines.
+
+## Migrations
+- Added Alembic revision `20241116_upgrade_roadmaps_and_user_repo_state.py` (down_revision `20241115_add_user_synced_repos`).
+- New columns have safe defaults (`0`, `false`, or nullable) to allow migration on existing data.
+
+## Notes / Next steps
+- No API or UI wiring yet; data can be populated later when GitHub fetch/sync logic is updated.
+- `progress_percent` currently unused; kept within 0–100 expectation but not enforced yet (could add a CHECK later if needed).
+
+## How to apply
+```bash
+cd commitly-backend
+alembic upgrade head
+```

SEARCH_FILTERS_IMPLEMENTATION.md

@@ -0,0 +1,205 @@
+# Search Page: Filters, Pagination & UI Integration
+
+**Date**: 2025-11-16  
+**Branch**: `feature/search-filters-pagination`  
+**Feature**: Complete search experience with filters, sorting, pagination, and UI integration
+
+## Overview
+This document tracks the implementation of comprehensive search functionality for the Commitly platform, including backend filter support and frontend UI integration.
+
+## Requirements
+
+### Backend
+- Extend `GET /api/v1/roadmap/catalog` endpoint with filters:
+  - `language`: Filter by programming language
+  - `tag`: Filter by topics/tags
+  - `difficulty`: Filter by difficulty level
+  - `min_rating`: Minimum average rating
+  - `min_views`: Minimum view count
+  - `min_syncs`: Minimum sync count
+- Add sorting options (already implemented: newest, most_viewed, most_synced, highest_rated, trending)
+- Implement proper pagination with filters
+
+### Frontend
+- Add filter controls for:
+  - Language (dropdown/multi-select)
+  - Tags/topics
+  - Difficulty
+  - Min rating, min views, min syncs (sliders/inputs)
+- Add pagination controls (page numbers/next/prev)
+- Wire query params → API calls → state
+- Separate sections for "Your Repositories" vs "Public Repositories"
+- Clean, non-cluttered UX
+
+## Implementation Plan
+
+### Phase 1: Backend Filter Implementation
+1. Update `RoadmapResultStore.list_catalog()` to accept filter parameters
+2. Modify SQL query to apply filters dynamically
+3. Update `RoadmapService.list_catalog()` to pass through filters
+4. Update API endpoint to accept filter query parameters
+5. Create Pydantic models for filter validation
+
+### Phase 2: Frontend UI Components
+1. Create filter UI components
+2. Add pagination controls
+3. Wire query parameters to API calls
+4. Update state management for filters
+5. Improve UX with loading states
+
+### Phase 3: Testing
+1. Add backend unit tests for filters
+2. Add integration tests for catalog endpoint
+3. Test pagination with various filter combinations
+4. Test UI responsiveness and error handling
+
+## Changes Made
+
+### Backend Changes
+
+#### Models (app/models/roadmap.py)
+- **CatalogFilters**: Pydantic model for filter parameters
+  - `language`: Optional[str] - Filter by programming language
+  - `tag`: Optional[str] - Filter by topic/tag
+  - `difficulty`: Optional[str] - Filter by difficulty level (beginner/intermediate/advanced)
+  - `min_rating`: Optional[float] - Minimum average rating (1.0-5.0)
+  - `min_views`: Optional[int] - Minimum view count
+  - `min_syncs`: Optional[int] - Minimum sync count
+  - `sort`: str - Sort order (newest, most_viewed, most_synced, highest_rated, trending)
+  - `page`: int - Page number (default 1)
+  - `page_size`: int - Items per page (default 20, max 100)
+
+- **CatalogPage**: Pydantic model for paginated response
+  - `items`: list[RoadmapResponse] - List of roadmap items for current page
+  - `page`: int - Current page number
+  - `page_size`: int - Number of items per page
+  - `total_count`: int - Total number of items matching filters
+  - `total_pages`: int - Total number of pages
+
+#### Repository Layer (app/services/roadmap_repository.py)
+- **RoadmapResultStore.list_catalog()**:
+  - Queries GeneratedRoadmap table with is_public=True
+  - Applies filters using JSON field queries:
+    - Language: `repo_summary["language"].astext == language`
+    - Tag: `repo_summary["topics"].contains([tag])`
+    - Difficulty: `repo_summary["difficulty"].astext == difficulty`
+    - Min rating: Calculates avg_rating and filters >= min_rating
+    - Min views: `repo_summary["view_count"].astext.cast(Integer) >= min_views`
+    - Min syncs: `repo_summary["sync_count"].astext.cast(Integer) >= min_syncs`
+  
+  - Implements 5 sorting options:
+    - **newest**: ORDER BY created_at DESC
+    - **most_viewed**: ORDER BY view_count DESC
+    - **most_synced**: ORDER BY sync_count DESC  
+    - **highest_rated**: ORDER BY (rating_sum / rating_count) DESC
+    - **trending**: ORDER BY (view_count * 0.4 + sync_count * 0.3 + avg_rating * 6 * 0.3) DESC
+  
+  - Pagination: Uses offset/limit based on page and page_size
+  - Returns: tuple of (list[RoadmapResponse], total_count)
+
+#### Service Layer (app/services/roadmap_service.py)
+- **RoadmapService.list_catalog()**:
+  - Simple wrapper that passes all parameters to repository store
+  - Returns: tuple of (list[RoadmapResponse], total_count)
+
+#### API Layer (app/api/roadmap.py)
+- **GET /catalog**: Updated to accept filter query parameters
+  - Query Parameters:
+    - page: int (default 1, >= 1)
+    - page_size: int (default 20, 1-100)
+    - language: Optional[str]
+    - tag: Optional[str]
+    - difficulty: Optional[str]
+    - min_rating: Optional[float] (1.0-5.0)
+    - min_views: Optional[int] (>= 0)
+    - min_syncs: Optional[int] (>= 0)
+    - sort: str (default "newest")
+  
+  - Response: CatalogPage with items, pagination metadata
+  - Calculates total_pages: math.ceil(total_count / page_size)
+
+#### Tests (tests/test_catalog_filters.py)
+Created comprehensive test suite with 22 test cases:
+
+**TestCatalogFilters** (8 tests):
+- test_list_catalog_no_filters: Basic listing without filters
+- test_list_catalog_language_filter: Filter by programming language
+- test_list_catalog_tag_filter: Filter by topic tag
+- test_list_catalog_difficulty_filter: Filter by difficulty
+- test_list_catalog_min_rating_filter: Filter by minimum rating
+- test_list_catalog_min_views_filter: Filter by minimum views
+- test_list_catalog_min_syncs_filter: Filter by minimum syncs
+- test_list_catalog_combined_filters: Multiple filters combined
+
+**TestCatalogSorting** (5 tests):
+- test_sort_by_newest: Verify created_at DESC ordering
+- test_sort_by_most_viewed: Verify view_count DESC ordering
+- test_sort_by_most_synced: Verify sync_count DESC ordering
+- test_sort_by_highest_rated: Verify avg_rating DESC ordering
+- test_sort_by_trending: Verify trending score calculation and ordering
+
+**TestCatalogPagination** (5 tests):
+- test_pagination_first_page: First page results
+- test_pagination_second_page: Second page has different items
+- test_pagination_beyond_last_page: Empty results beyond last page
+- test_pagination_page_size_consistency: Page size is respected
+- test_pagination_total_count_consistency: Total count consistent across pages
+
+**TestCatalogEdgeCases** (4 tests):
+- test_empty_results: Handle empty result sets
+- test_invalid_page_number: Handle edge case page numbers
+- test_min_rating_boundary: Boundary values for min_rating
+- test_filter_with_sorting: Filters work correctly with sorting
+
+#### Code Quality
+- ✅ All flake8 linting issues resolved
+- ✅ Code formatted with black
+- ✅ Imports sorted with isort --profile black
+- ✅ Line length within 88 character limit
+- ✅ No whitespace issues
+
+### Frontend Changes
+
+#### API Service (lib/services/repos.ts)
+- **Updated Type Definitions**:
+  - Added `CatalogFilters` type with 9 optional filter parameters
+  - Added `CatalogPage` type matching backend pagination response
+  - Includes: items[], page, page_size, total_count, total_pages
+
+- **Updated listCatalog() Method**:
+  - Now accepts optional `CatalogFilters` parameter
+  - Builds URL query string from filter parameters
+  - Returns `CatalogPage` instead of array
+  - Maintains backward compatibility with no filters
+
+#### Provider (components/providers/roadmap-catalog-provider.tsx)
+- **Updated Data Fetching**:
+  - Modified to handle `CatalogPage.items` response structure
+  - Extracts items from paginated response
+  - Maintains existing synced/pending/loading state management
+
+#### Search Page (app/search/page.tsx)
+- **Current State**: Using existing provider data
+- **Future Enhancements** (not implemented yet):
+  - Add filter UI components for all 6 filter types
+  - Add pagination controls (page navigation)
+  - Wire filters to URL query params
+  - Add sorting dropdown
+  - Show total results count
+  - Add loading/empty states for filtered results
+
+**Note**: The search page currently uses the existing provider pattern which now supports the paginated API. Advanced filtering UI (dropdowns, sliders, etc.) can be added as a future enhancement.
+
+### Database Changes
+- No schema changes required (using existing columns from previous prompts)
+
+## Testing Strategy
+- Unit tests for filter logic
+- Integration tests for catalog endpoint
+- End-to-end tests for search page
+- Manual testing of UX flow
+
+## Notes
+- Leveraging existing metadata columns added in Prompt 1
+- No social features or complex tracking
+- Focus on clean, simple catalog/search experience

commitly-backend/app/api/roadmap.py

@@ -1,21 +1,20 @@
 from __future__ import annotations
 
-from typing import Annotated, Optional
+from typing import Optional
 
-from fastapi import APIRouter, Depends, HTTPException, Response, status
+from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
 from sqlalchemy.orm import Session
 
 from app.core.auth import ClerkClaims, optional_clerk_auth, require_clerk_auth
 from app.core.database import get_db
 from app.models.roadmap import (
+    CatalogPage,
     RatingRequest,
     RatingResponse,
-    RoadmapCatalogPage,
     RoadmapRequest,
     RoadmapResponse,
     UserRepoStateResponse,
 )
-from app.services.roadmap_repository import SortOption
 from app.services.roadmap_service import RoadmapService, build_roadmap_service
 
 router = APIRouter()
@@ -36,27 +35,50 @@ def get_user_id(claims: ClerkClaims) -> str:
     return user_id
 
 
-@router.get("/catalog", response_model=RoadmapCatalogPage)
+@router.get("/catalog", response_model=CatalogPage)
 async def list_roadmaps(
-    page: int = 1,
-    page_size: int = 20,
-    sort: Annotated[
-        SortOption,
-        "Sort option: newest, most_viewed, most_synced, highest_rated, trending",
-    ] = "newest",
+    page: int = Query(1, ge=1, description="Page number"),
+    page_size: int = Query(20, ge=1, le=100, description="Items per page"),
+    language: str | None = Query(None, description="Filter by programming language"),
+    tag: str | None = Query(None, description="Filter by topic/tag"),
+    difficulty: str | None = Query(None, description="Filter by difficulty"),
+    min_rating: float | None = Query(
+        None, ge=1.0, le=5.0, description="Minimum average rating"
+    ),
+    min_views: int | None = Query(None, ge=0, description="Minimum view count"),
+    min_syncs: int | None = Query(None, ge=0, description="Minimum sync count"),
+    sort: str = Query(
+        "newest",
+        description=(
+            "Sort order: newest, most_viewed, most_synced, " "highest_rated, trending"
+        ),
+    ),
     service: RoadmapService = Depends(get_roadmap_service),
-) -> RoadmapCatalogPage:
-    """
-    List roadmaps with pagination and sorting.
-
-    Sort options:
-    - newest: Order by updated_at DESC (default)
-    - most_viewed: Order by view_count DESC
-    - most_synced: Order by sync_count DESC
-    - highest_rated: Order by average rating DESC
-    - trending: Order by trending score (combines views, syncs, and ratings)
-    """
-    return await service.list_catalog(page, page_size, sort)
+) -> CatalogPage:
+    """List catalog with filters, sorting, and pagination."""
+    import math
+
+    items, total_count = await service.list_catalog(
+        page=page,
+        page_size=page_size,
+        language=language,
+        tag=tag,
+        difficulty=difficulty,
+        min_rating=min_rating,
+        min_views=min_views,
+        min_syncs=min_syncs,
+        sort=sort,
+    )
+
+    total_pages = math.ceil(total_count / page_size) if total_count > 0 else 0
+
+    return CatalogPage(
+        items=items,
+        page=page,
+        page_size=page_size,
+        total_count=total_count,
+        total_pages=total_pages,
+    )
 
 
 @router.get("/cached/{owner}/{repo}", response_model=RoadmapResponse)

commitly-backend/app/models/roadmap.py

@@ -155,6 +155,34 @@ class RoadmapRequest(BaseModel):
     )
 
 
+class CatalogFilters(BaseModel):
+    """Filters for roadmap catalog search."""
+
+    language: Optional[str] = Field(None, description="Filter by programming language")
+    tag: Optional[str] = Field(None, description="Filter by topic/tag")
+    difficulty: Optional[Literal["beginner", "intermediate", "advanced"]] = Field(
+        None, description="Filter by difficulty level"
+    )
+    min_rating: Optional[float] = Field(
+        None, ge=1.0, le=5.0, description="Minimum average rating"
+    )
+    min_views: Optional[int] = Field(None, ge=0, description="Minimum view count")
+    min_syncs: Optional[int] = Field(None, ge=0, description="Minimum sync count")
+    sort: Optional[
+        Literal["newest", "most_viewed", "most_synced", "highest_rated", "trending"]
+    ] = Field("newest", description="Sort order")
+
+
+class CatalogPage(BaseModel):
+    """Paginated catalog response."""
+
+    items: List["RoadmapResponse"]
+    page: int = Field(ge=1, description="Current page number")
+    page_size: int = Field(ge=1, le=100, description="Number of items per page")
+    total_count: int = Field(ge=0, description="Total number of items")
+    total_pages: int = Field(ge=0, description="Total number of pages")
+
+
 class TimelineResource(BaseModel):
     label: str
     href: Annotated[str, Field(max_length=500)]