Conversation
Greptile OverviewGreptile SummaryAdds comprehensive Diavgeia decision integration to OpenCouncil, allowing admins to link official government decisions to meeting subjects. Introduces a new Key changes:
Important notes:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Admin as Admin User
participant UI as DecisionsPanel
participant API as /api/decisions
participant Task as pollDecisions
participant Backend as Task Server
participant Diavgeia as Diavgeia API
participant DB as Database
Note over Admin,DB: Manual Decision Linking Flow
Admin->>UI: Open Decisions Panel
UI->>API: GET /decisions
API->>DB: getDecisionsForMeeting()
DB-->>API: Return existing decisions
API-->>UI: Display decisions by subject
Admin->>UI: Fill form (pdfUrl, ada, etc.)
UI->>API: PUT /decisions
API->>DB: Verify subject ownership
DB-->>API: Subject validated
API->>DB: upsertDecision(subjectId, data, createdById)
DB-->>API: Decision created/updated
API-->>UI: Success
UI-->>Admin: Show linked decision
Note over Admin,DB: Automated Polling Flow
Admin->>UI: Click "Poll Decisions"
UI->>Task: requestPollDecisions(cityId, meetingId)
Task->>DB: Fetch meeting + subjects
DB-->>Task: Meeting data with subjects
Task->>DB: Create TaskStatus record
DB-->>Task: Task created
Task->>Backend: POST to TASK_API_URL
Backend-->>Task: Task queued
Task-->>UI: Task started
UI-->>Admin: Show "polling in progress"
Note over Backend,DB: Background Processing
Backend->>Diavgeia: Search decisions (by date, unitId)
Diavgeia-->>Backend: Return decision list
Backend->>Backend: Match decisions to subjects
Backend->>API: POST /api/taskStatuses/{id} (callback)
API->>Task: handlePollDecisionsResult(result)
Task->>DB: Validate subjectIds belong to meeting
DB-->>Task: Validation complete
loop For each matched decision
Task->>DB: upsertDecision(subjectId, ada, taskId)
DB-->>Task: Decision linked
end
Task-->>API: Processing complete
API-->>Backend: Success
Note over Admin,DB: View Updated Decisions
Admin->>UI: Refresh panel
UI->>API: GET /decisions
API->>DB: getDecisionsForMeeting()
DB-->>API: Decisions (with task/user source)
API-->>UI: Display with source badges
UI-->>Admin: Show automated + manual decisions
|
prisma/migrations/20260202164839_add_diavgeia_decisions/migration.sql
Outdated
Show resolved
Hide resolved
Additional Comments (1)
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/components/cities/CityForm.tsx
Line: 54:61
Comment:
**Missing diavgeiaUnitId in type**
`CityForm`’s local `administrativeBodies` state type doesn’t include `diavgeiaUnitId`, but the API now returns it and `AdministrativeBodiesList` expects it. TypeScript will fail when calling `setAdministrativeBodies(data)` (data includes extra field) / passing `administrativeBodies` to `AdministrativeBodiesList`.
```suggestion
const [administrativeBodies, setAdministrativeBodies] = useState<Array<{
id: string;
name: string;
name_en: string;
type: AdministrativeBodyType;
youtubeChannelUrl?: string | null;
notificationBehavior?: NotificationBehavior | null;
diavgeiaUnitId?: string | null;
}>>([])
```
How can I resolve this? If you propose a fix, please make it concise. |
852e96c to
b4b3933
Compare
🚀 Preview deployment ready!Preview URL: https://pr-193.preview.opencouncil.gr The preview will be automatically updated when you push new commits. This preview uses an isolated database with seed data (migrations were applied). |
|
I force-pushed to address bot review comment(s) (1, 2):
|
b4b3933 to
b989920
Compare
b989920 to
dca52ef
Compare
|
I force-pushed to address bot review comment(s) (1, 2):
|
dca52ef to
4d6d5de
Compare
4d6d5de to
e2f4b32
Compare
e2f4b32 to
ee3ce73
Compare
|
I force-pushed to address bot review comment(s) (1, 2):
|
ee3ce73 to
a3efa59
Compare
412cfb6 to
84ab207
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
84ab207 to
9e94737
Compare
|
I force-pushed to address bot review comment(s) (1, 2):
|
Add support for linking meeting subjects to decisions from the Greek Government Transparency portal (Diavgeia). Schema changes: - Add Decision model with ada, protocolNumber, title, pdfUrl, issueDate - Add City.diavgeiaUid for organization identifier - Add AdministrativeBody.diavgeiaUnitId for unit filtering UI additions: - Add DecisionsPanel for managing decision links in meeting admin - Add diavgeiaUid field to CityForm - Add diavgeiaUnitId field to AdministrativeBodiesList - Add export functionality for manual matching Task integration: - Add pollDecisions task type and handler - Register in task registry for automatic callback handling - Support filtering by diavgeiaUnitId for more accurate matching Also update CLAUDE.md with guidance on checking for existing types before creating new ones.
Adds event propagation stop to DropdownMenuContent to prevent infinite focus loops when used inside Dialogs. Mirrors the fix applied to SelectContent in 889da63.
Display linked Diavgeia decisions as a collapsible card on the subject page, showing the title, ADA, protocol number, issue date, and a link to the PDF.
Add CRON_SECRET env var and GET /api/cron/poll-decisions endpoint that polls Diavgeia for decisions on recent meetings (last 90 days) with unlinked subjects. Processes up to 10 meetings per invocation. Authenticated via Bearer token.
Add requestPollDecisionForSubject() server action that any user can trigger from the subject page, with 5-minute rate limiting. Show a 'Fetch from Diavgeia' button when no decision is linked, with loading state and confirmation message.
Document the team convention for using fixup! commits when modifying existing branch commits, and autosquash rebase to fold them in.
Add automated task initiation section to task-architecture.md covering the Diavgeia decision polling cron endpoint, setup instructions, and pattern for adding new cron tasks. Add CRON_SECRET to env docs and .env.example.
Add /admin/diavgeia page with summary stat cards (total discoveries, meetings still polling, avg discovery/publish delays), backoff schedule table, and sortable discoveries table. Includes progressive backoff for cron polling and a stats API endpoint for monitoring.
Add getPollingHistoryForMeeting() to derive current backoff tier and next eligible poll time. DecisionsPanel now shows poll count, start date, tier label, and next auto-poll date below the toolbar.
Add decision counts column (X/Y linked/eligible) to the meetings table and a Decisions card in expanded row content with lazy-loaded polling history (tier, last poll, next eligible poll).
Standalone script to link cities and administrative bodies to their Diavgeia counterparts by matching names with accent-stripped comparison, substring matching, and known abbreviations (ΔΣ, ΔΕ). Supports --dry-run to preview and --force to overwrite existing UIDs.
Allows connecting to an opencouncil-tasks preview deployment for a given PR number. Starts an ngrok tunnel so the remote tasks server can POST callbacks back to localhost, and overrides NEXTAUTH_URL to the ngrok URL for correct URL construction.
Click the Eye button on any poll row to open a Sheet sidebar showing full metadata (status, time, city, meeting, results summary), a link to the meeting admin page, and copyable request/response JSON bodies.
Two failure paths in startTask and handleTaskUpdate were setting status to 'failed' without storing the error in responseBody, making it impossible to debug failures from the admin UI.
Add server-side URL param filters to the Diavgeia admin polling stats page. City filter narrows polls by city; meeting filter appears once a city is selected and narrows further by council meeting ID. Changing the city automatically clears the meeting selection. Also adds updateParams() to useUrlParams hook for atomic multi-param URL updates.
Connects to the database used by an opencouncil preview deployment. Detects whether the PR has an isolated DB (SSH tunnel) or uses the shared staging DB (reads URL from server), and forces external DB mode to skip local postgres. Requires OC_PREVIEW_SSH to be set to the preview server SSH target. Can be used independently or alongside --preview-tasks=M.
…a admin Extract getBackoffState() helper from inline tier calculation logic so it can be reused. Expand getPollingStats() to return per-meeting details (unlinked subjects, backoff state, poll history) instead of just a count. Add a collapsible table showing each still-polling meeting with city, date, unlinked/eligible counts, first/last poll dates (with timestamp tooltips on hover), backoff tier, and a details sidebar listing the unlinked subject names with a link to the meeting admin page. Refactor getPollingHistoryForMeeting() to use the shared helper.
9e94737 to
4f3bbf6
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| currentTierLabel, | ||
| nextPollEligible, | ||
| }; | ||
| }); |
There was a problem hiding this comment.
Diavgeia stats filters ignored for discoveries and meetings
Medium Severity
getPollingStats accepts cityId and councilMeetingId but never applies them to the discoveries query, discoveryDetails, stillPollingMeetings, or meetingsStillPolling. Only recentPollTasks and pollMeetings are filtered. As a result, when a city or meeting is selected in the Diavgeia admin UI, the Discoveries table, Meetings Still Polling table, and summary stats (total discoveries, discovery delay, publish delay) still show global data instead of the filtered subset.
| } | ||
|
|
||
| console.log(`Poll decisions completed: ${processedCount} processed, ${result.unmatchedSubjects.length} unmatched, ${result.ambiguousSubjects.length} ambiguous`); | ||
| } |
There was a problem hiding this comment.
handlePollDecisionsResult crashes on malformed task result
Medium Severity
handlePollDecisionsResult assumes result.matches, result.unmatchedSubjects, and result.ambiguousSubjects are always defined. If the task server sends a malformed callback (e.g. result: {} or result: { matches: undefined }), the handler throws on result.matches.map, for...of result.matches, or result.unmatchedSubjects.length, causing a 500 and failed callback processing.


Fixes #103
Decisions polled from Diavgeia (support for manual addition)

Subject's Page now shows the Decision iff available
Allows to manually fetch if Decision not available

Admin dashboard for Polling Stats

Note
Medium Risk
Introduces a new persisted
Decisionmodel with Prisma migration and multiple new polling/management endpoints (cron + admin + public trigger), so regressions could affect meeting/subject data integrity and background task load. Risk is mitigated by auth checks on admin routes andCRON_SECRETgating for cron, but rollout requires correct env configuration and monitoring.Overview
Adds end-to-end support for linking meeting
Subjects to official Diavgeia decisions, including a newDecisiontable (1:1 withSubject) plus configuration fieldsCity.diavgeiaUidandAdministrativeBody.diavgeiaUnitIds.Implements automated polling via new cron routes
GET /api/cron/poll-decisionsandGET /api/cron/poll-decisions-stats(BearerCRON_SECRET), plus admin/public workflows to fetch and manage decisions: an adminDecisionsPanel, a meeting decisions API (GET/PUT/DELETE /api/cities/[cityId]/meetings/[meetingId]/decisions), decision status surfacing in admin meetings lists, and decision display + on-demand fetch on the subject page.Adds a new
/admin/diavgeiapolling stats dashboard, expands env/docs (CRON_SECRET, preview DB SSH), updates Nix dev tooling with--preview-db/--preview-tasks(ngrok + SSH tunneling), and includes small UI stability tweaks (dropdown focus handling, hash-linkedCollapsibleCard).Written by Cursor Bugbot for commit 4f3bbf6. This will update automatically on new commits. Configure here.
Greptile Summary
This PR adds comprehensive support for linking council meeting subjects to their official Diavgeia decisions (Greek government transparency platform). The implementation includes:
Database Schema: New
Decisiontable (1:1 withSubject) stores decision metadata (ada,protocolNumber,title,pdfUrl,issueDate). Configuration fields added:City.diavgeiaUidandAdministrativeBody.diavgeiaUnitIdsfor Diavgeia API matching.Automated Polling: Cron job (
/api/cron/poll-decisions) runs 2x/day to poll recent meetings (90-day window) for decisions. Uses progressive backoff schedule (daily→weekly) to reduce load as meetings age. Rate limiting prevents duplicate concurrent tasks viaTaskStatusqueries filtering non-terminal states.Manual Workflows: Admins can bulk-manage decisions via DecisionsPanel UI and
/api/cities/[cityId]/meetings/[meetingId]/decisionsAPI. Public users can trigger on-demand polling from subject pages with simple 5-minute rate limiting. All mutations validate subject ownership within the meeting before upserting.Monitoring: New
/admin/diavgeiadashboard displays polling effectiveness stats (discovery delays, publish delays, meetings still polling) to tune the backoff schedule. Task error handling improved to persist diagnostic messages inresponseBody.Dev Tooling: Nix dev runner enhanced with
--preview-dband--tasks-previewflags to connect to remote preview environments via ngrok tunnels. Import script provided to seed Diavgeia configuration from external sources.All previous thread concerns have been addressed: auth is properly enforced via
withUserAuthorizedToEdit, upsert is idempotent onsubjectId, PostgreSQL UNIQUE on nullableadaworks correctly, rate limiting covers all non-terminal task states, and subject validation prevents cross-meeting pollution.Confidence Score: 4/5
CRON_SECRETis optional, so cron endpoints will return 503 until configured in production, (2) backoff schedule effectiveness depends on real-world Diavgeia publishing patterns and may need tuning, (3) the 60-second task status polling loop on subject pages creates sustained load during user-triggered polls but is rate-limited.CRON_SECRETis configured in production environment before first cron run. Monitor/admin/diavgeiastats after deployment to verify backoff schedule matches actual Diavgeia publishing delays.Important Files Changed
Decisiontable with 1:1 relation toSubject, plusCity.diavgeiaUidandAdministrativeBody.diavgeiaUnitIdsfields for Diavgeia integrationDecisiontable with unique indices onsubjectIdandada, adds Diavgeia configuration fields to City and AdministrativeBodyCRON_SECRETbearer token, dispatches polling for recent meetings viapollDecisionsForRecentMeetingssubjectId, preserves originaltaskId/createdByIdon updatesresponseBodyfor failed tasks instead of losing diagnostic infoSequence Diagram
sequenceDiagram participant Cron as Cron Job participant API as Poll API participant DB as Database participant Tasks as Task Backend participant Admin as Admin UI participant User as Subject Page Note over Cron,Tasks: Automated Polling Flow Cron->>API: GET /api/cron/poll-decisions API->>DB: Find meetings (90d, unlinked decisions) DB-->>API: Recent meetings list API->>API: Apply backoff filter API->>DB: Create TaskStatus (pollDecisions) API->>Tasks: POST task to TASK_API_URL Tasks-->>API: Task accepted API-->>Cron: Result (dispatched count) Note over Tasks,DB: Task Processing Tasks->>Tasks: Fetch Diavgeia API Tasks->>API: POST /api/taskStatuses/{id} API->>DB: Validate subjectIds API->>DB: Upsert Decision records DB-->>API: Success API-->>Tasks: Callback acknowledged Note over Admin,User: Manual Workflows Admin->>API: PUT /api/cities/{id}/meetings/{id}/decisions API->>DB: Validate subject ownership API->>DB: Upsert decision (createdById) DB-->>Admin: Decision saved User->>API: requestPollDecisionForSubject API->>DB: Check rate limit (5min) API->>DB: Create TaskStatus API->>Tasks: POST task User->>User: Poll taskStatus (60s loop) User->>API: GET /api/cities/{id}/meetings/{id}/taskStatuses/{id} API-->>User: Task statusLast reviewed commit: 4f3bbf6