[LFXV2-989] Add handlers for voting and survey data from DynamoDB#27
Merged
[LFXV2-989] Add handlers for voting and survey data from DynamoDB#27
Conversation
Implement comprehensive handlers for processing ITX voting and survey data from DynamoDB tables. The handlers use a two-tier architecture: Database structs with string fields for unmarshaling Meltano data, then conversion to Input structs with proper types for indexer/FGA. Voting handlers: - convertMapToInputVote: Converts polls with committee UID mapping - convertMapToInputIndividualVote: Converts individual votes with ranked choice answer handling - String-to-int conversion for all numeric fields Survey handlers: - convertMapToInputSurvey: Converts surveys with nested committee data - convertMapToInputSurveyResponse: Converts survey responses - Proper type conversion for all numeric metrics fields Both handlers include proper NATS message publishing to indexer and fga-sync topics with committee/project UID lookups from KV buckets. Updated Meltano configuration: - Added nkeys dependency to target-nats-kv - Set empty replication keys for tables without timestamp fields 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Andres Tobon <andrest2455@gmail.com>
Update access message handlers to conditionally include relations and references only when the corresponding fields are non-empty. Also skip sending access messages entirely when there are no relations or references to send, avoiding unnecessary empty messages to NATS. Changes: - Vote access: Only include project/committee relations if UIDs exist - Individual vote access: Only include username relations and project/vote references if values exist - Survey access: Only include committee/project references if UIDs exist - Survey response access: Only include username relations and survey/project/committee references if values exist All handlers now check if there are any relations or references before publishing the access message to NATS. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Andres Tobon <andrest2455@gmail.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive handlers for processing ITX voting and survey data from DynamoDB tables in the v1-to-v2 sync helper service. It implements data processing for polls, individual votes, surveys, and survey responses, syncing them from the v1 system to the v2 indexer and FGA services.
Changes:
- Added handlers for voting data (polls and individual votes) with committee UID mapping and ranked choice answer handling
- Added handlers for survey data (surveys and survey responses) with nested committee data and user associations
- Configured Meltano to sync four new DynamoDB tables:
itx-poll,itx-poll-vote,itx-surveys, anditx-survey-responses
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| meltano/meltano.yml | Added four new DynamoDB tables to sync configuration with empty replication keys |
| meltano/load/target-nats-kv/pyproject.toml | Added nkeys dependency for NATS key-value store operations |
| go.mod | Added lfx-v2-indexer-service as an indirect dependency |
| go.sum | Updated with checksums for the new indexer service dependency |
| cmd/lfx-v1-sync-helper/models_voting.go | Defined voting data structures with separate DB and input models for proper type conversion |
| cmd/lfx-v1-sync-helper/models_surveys.go | Defined survey data structures with separate DB and input models for proper type conversion |
| cmd/lfx-v1-sync-helper/handlers_voting.go | Implemented vote and individual vote handlers with indexer and FGA message publishing |
| cmd/lfx-v1-sync-helper/handlers_survey.go | Implemented survey and survey response handlers with indexer and FGA message publishing |
| cmd/lfx-v1-sync-helper/handlers.go | Registered the new table handlers in the main KV put handler |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add validation to skip vote sync when parent project mapping doesn't exist - Fix username mapping in survey response handler using mapUsernameToAuthSub - Add SES email tracking fields to survey response conversion 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Andres Tobon <andrest2455@gmail.com>
jordane
reviewed
Jan 16, 2026
…ndler context Add warning logs when string-to-integer conversions fail in voting and survey handlers. Use function-scoped logger with handler name to make logs more maintainable and avoid repetition. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Andres Tobon <andrest2455@gmail.com>
andrest50
force-pushed
the
andrest50/voting-survey-handlers
branch
from
63e0740 to
d97f9ac
Compare
jordane
previously approved these changes
Jan 16, 2026
Rename IndividualVoteInput to VoteResponseInput and update all related functions and constants to use vote_response naming for consistency with v2 system terminology. Fix FGA access configuration: - Vote: Use references instead of relations for project/committee - Vote response: Keep proper access control with vote reference - Survey response: Use owner relation and survey reference, fix AccessCheckObject Update indexer parent refs to use conditional appending instead of hardcoded template strings to avoid sending empty parent references. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Andres Tobon <andrest2455@gmail.com>
…elper into andrest50/voting-survey-handlers
jordane
approved these changes
Jan 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
LFXV2-989
Summary
Implement comprehensive handlers for processing ITX voting and survey data from DynamoDB tables. This adds support for syncing polls, individual votes, surveys, and survey responses from the v1 system to the v2 indexer and FGA services.
Key Features
Voting Handlers:
handleVoteUpdate: Processes poll data with committee UID mappinghandleIndividualVoteUpdate: Processes individual votes with ranked choice answer handlingSurvey Handlers:
handleSurveyUpdate: Processes surveys with nested committee data and metricshandleSurveyResponseUpdate: Processes survey responses with user associationsArchitecture
Uses a two-tier struct pattern for proper type handling:
PollDB,VoteDB,SurveyDatabase,SurveyResponseDatabase)InputVote,IndividualVoteInput,SurveyInput,SurveyResponseInput)FGA Access Control
All handlers implement conditional access message publishing:
Meltano Configuration
nkeysdependency to target-nats-kv🤖 Generated with Claude Code