Fix: Remove trailing paragraph from reports stored in database by Jamerson-santos · Pull Request #2201 · AletheiaFact/aletheia

Jamerson-santos · 2026-01-27T14:19:43Z

Description

This PR addresses the technical debt tracked in the issue by fixing the root cause of trailing <p> paragraphs being persisted in report editor content stored in the database.

Previously, the frontend had to apply a temporary workaround (e.g., removeTrailingParagraph) to sanitize API responses before loading them into the editor, because some reports were saved with an extra trailing <p> node. This PR prevents saving reports with trailing paragraphs at the persistence layer and includes a data cleanup to normalize existing records.

As a result:

Reports are stored without trailing paragraphs
Frontend filtering/workaround is no longer required
The editor works correctly with TrailingNodeExtension
Existing reports remain compatible after cleanup/migration

Related Ticket #2179

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Existing feature enhancement (non-breaking change which modifies existing functionality)

Testing

To validate this change:

Backend / Data validation

Create or update a report and save it.
Verify the persisted editor content in the database no longer contains a trailing <p> paragraph.
Fetch the report via the API and confirm the returned content does not include an extra trailing paragraph.

Backward compatibility

Test loading reports created before this fix (including reports that previously contained trailing <p> elements).
Confirm the editor loads successfully and allows inserting new nodes (e.g., questions) normally.

Editor regression checks (core workflow)

Go to the Claim page.
Click on any sentence to open the drawer.
In “Which questions should the verification answer?”, try adding/removing/reinserting question nodes multiple times.
Confirm nodes are inserted at the correct position and no nested/duplicate trailing paragraphs appear in the resulting HTML structure.

Impacted scenarios:

Saving report editor content
Loading editor content from API
Compatibility with historical reports
Interaction with TrailingNodeExtension and insertion logic for new nodes

No special build is required beyond the standard environment. If a migration/script is included, ensure it runs in the target environment before validating.

Developer Checklist

General

Code is appropriately commented, particularly in hard-to-understand areas
Repository documentation has been updated (Readme.md) with additional steps required for a local environment setup.
No console.log or related logging is added.
No code is repeated/duplicated in violation of DRY.
Documented with TSDoc all library and controller new functions

Frontend Changes

No new styling is added through CSS files (Unless it's a bugfix/hotfix)
All types are added correctly

Backend Changes

All endpoints are appropriately secured with Middleware authentication
All new endpoints have a interface schema define

Tests

All existing unit and end to end tests pass across all services
Unit and end to end tests have been added to ensure backend APIs behave as expected

Test IDs

Include the test ID when adding new tasks or components.
Check that test IDs are present in the modified components.

Merge Request Review Checklist

An issue is linked to this PR and these changes meet the requirements outlined in the linked issue(s)
High risk and core workflows have been tested and verified in a local environment.
Enhancements or opportunities to improve performance, stability, security or code readability have been noted and documented in Project do Github issues if not being addressed.
Any dependent changes have been merged and published in downstream modules
Changes to multiple services can be deployed in parallel and independently. If not, changes should be broken out into separate merge requests and deployed in order.

src/components/Collaborative/VisualEditor.style.tsx

migrations/20260114110506-remove-trailing-paragraph-from-review-tasks.ts

src/components/Collaborative/VisualEditor.style.tsx

src/machines/reviewTask/actions.ts

…tor styles

sonarqubecloud · 2026-01-29T16:05:30Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

migrations/20260114110506-remove-trailing-paragraph-from-review-tasks.ts

@caneppelevitor

* Update verificationRequest.json * refactor(verification-request): extract date filter to util and reduce duplication * remove unnecessary imports * poc: dashboard cop30 * refactor(dashboard): Move component to parent for context access * add filter by data range after rebase * Fix: handleResetFilters * feat(filters): apply inclusive date range by normalizing start and end times * style(filters): add responsive margin to DateRangePicker on small screens * feat(cop30): localize COP30 section * poc: Dashboard for verification request * Resolved build error * updating sintax import and deleting identical phrase * Adding new dashboard * Refactor and add toggle button * Finishing the Verification Requests Dashboard UI * Sonar cloud: Resolve some comments * add migrate * Removing telegram from enum * Fixing unnecessary migrations * security: Add guard to ensure totalCount is not zero * Creating an endpoint to retrieve COP30 topics and making the dashboard dynamic * chore: add down function to migration for rollback safety * Updating reviews related to the COP30 topic, styling the dashboard, and correcting sonar requests * tracking page: MVP * feat(tracking): rename targetId to verificationRequestId and use history _id as key * refactor(controller, service): better typing and error handling * Refactor(tracking): Optimizes the API response for Stepper * Creating a type for "getByTargetIdModelAndType" * Fix const naming and typing file path * Refactor(api-client): reusing history endpoint to create page * feat(ui/tracking): Implement core visual design for status stepper * chore: resolve sonar comment * Making URL_PATTERN less rigid for forms (#2136) * Making the pattern less rigid * check URLs with dot at the end * Feat: Wikidata Aliases & Topic Timestamps (#2138) * feat: topic search base on alias and schema enhanced - Added alias on topic autocomplete option passing mui filtering - Added alias on topic schema for future alias search on DB - Added timestamps on topic schema - Alias and timestamps migrations * fix: add includeAliases parameter to WikidataService.queryWikibaseEntities() * feat: enables search topics by alias and enhance topics search * feat: migration special case for cop30 topic * fix: sonarcloud sort operation to a separate statement and String.raw * feat: enhancing functions params with expected param types * fix: rolling back searchTopics changes preventing memory issues * feat: error handling on migration * fix: add TypeScript types and null safety to AdvancedSearch component and extracting defaultValue into normalizedDefaultValue function * fix: add TypeScript types and optimize topic search performance - Add type annotations and interfaces across backend services - Replace string literals with FilterType enum in ActiveFilters - Add null safety guards and refactor ChipComponent to reduce duplication - Rename wbentity to searchResult for clarity * feat: try catch on alias migration * fix: rolling back to regular string * fix: migrations coded wrong and with missing types * feat: add new typing interface for review, sentence, sentence topics and topic * feat(endpoint): add a new endpoint to retrieve all sentence stats related to Cop30 topics using the sentence module * feat: add endpoint stats, creating word translations for the topics, filtering topics from sentences involved with Cop30 and adding typing * FIX/CHORE Adding timestamps for modules (#2141) * fix: user agent for wikidata calls (#2111) * Release/v1.0.13 (#2120) * hotfix: filtering undefined personalities with not allowed wikidata instances * chore: added timestamps for schemas and migration to populate database * fix: address code review feedback on timestamp migration - Replace .toArray() with cursor-based batch processing to prevent memory issues on large collections - Add date validation with isValidDate() helper function to prevent Invalid Date objects - Process documents in batches of 1000 for better memory management - Add documentation explaining why original 'date' fields are kept - Apply batch processing to both up() and down() migrations Addresses review comments from @caneppelevitor on PR #2141 * chore: update collections --------- Co-authored-by: Vítor Caneppele <99989478+caneppelevitor@users.noreply.github.com> Co-authored-by: caneppelevitor <vitor.a.caneppele@gmail.com> * feat: creating a stats endpoint for cop30 topics, adding typing and changing the endpoints to sentenceApi * refactor: avoiding duplication in the cop30 topic filters * feat: updating section cop30 to retrieve stats values from the backend * fix: preventing review tasks without personality to break postprocess (#2155) * fix: preventing review tasks without personality to break postprocess * fix: sonarcloud complains * fix: code readability complain on lookup aggregations * fix: generateHref function for review task type image without personality and properly displaying images on claim card (#2158) * fix: generateHref function for review task type image without personality and properly displaying images on claim card * fix: properly displaying image on claim page on mobile and desktop resolutions * fix(topic): adjusting large topics with maxWidth to prevent card breakage temp * feat(endpoint): add IsPublic decorator and remove unnecessary typing * refactor(auth): replace deprecated IsPublic with Auth({ public: true }) * Fix: Input Component Overlapping on Reinsert * refactor(sentence): rename function, fix endpoint, and standardize classification to English * feat: add review button for images and fix SVG centralization * feat(ui): adjust the spacing of the Stats Cards to precent breakage on smaller screens * feat: add history button and ontegration with claim history route * Fix: Remove trailing paragraph from API response to prevent duplicate nodes * Change the home page SEO description * feat: add support for History route using TargetModel and optional parameter * fix(style): changing the color of the Verification Request card chip and also its value when it is undefined * feature: adds identified personalities on verification request drawer * fix: no sql injection protection on wikidataservice and code smells fix * feat: migrate high-priority admin endpoints to unified @Auth() decorator system This commit migrates all security-critical admin endpoints from the old inconsistent auth patterns to the new unified @Auth() decorator system, improving code maintainability and security. Changes: - Migrated 15 admin endpoints to use @adminonly() decorator - Replaced 20+ @ispublic() with @public() for consistency - Removed old patterns: @UseGuards(AbilitiesGuard), @CheckAbilities() - Updated imports across all affected controllers Controllers migrated: 1. claim-review.controller.ts (4/4 endpoints - COMPLETE) - Admin: PUT/DELETE review endpoints - Public: GET review endpoints 2. claim.controller.ts (15/39 endpoints - admin + public) - Admin: DELETE claim, PUT hidden status, GET debate editor - Public: 12 GET endpoints for claim pages 3. personality.controller.ts (7/16 endpoints - admin + public API) - Admin: PUT hidden status, DELETE personality - Public: 5 GET endpoints for personality data 4. daily-report.controller.ts (1/1 endpoint - COMPLETE) - Admin: POST daily report sending 5. badge.controller.ts (4/4 endpoints - COMPLETE) - Admin: POST/PUT badge management, GET admin page 6. name-space.controller.ts (4/4 endpoints - COMPLETE) - Admin: POST/PUT/GET namespace management New file: - server/auth/auth_migration_mapping.md Comprehensive tracking document mapping all 32 controllers and their migration status Benefits: ✅ Consistent auth pattern across all admin endpoints ✅ Improved code readability and maintainability ✅ Explicit security boundaries for critical operations ✅ No breaking changes - backward compatible ✅ Foundation for continued migration of remaining endpoints See auth_migration_mapping.md for detailed migration plan and progress. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: remove auth-migration-mapping.md * feat: migrate remaning deprecated decorators * refactor: overhaul dashboard architecture and localization logic - Frontend: Applied Atomic Design to dashboard components and implemented new interfaces - Localization: Shifted timeAgo strings to frontend JSON keys and removed redundant backend translations - Backend: Decoupled getStats into four modular functions and converted to a Promise-based structure - Schema: Created a Timestamp type and updated the VerificationRequest schema to improve 'Recent Activity' logic - UX: Added 'No data available' states for empty dashboard widgets * test(login):Adding login flow testing. * test(login): add the contains clause to ensure the user receives the appropriate feedback * test(login): fix logout test by registering intercept before action * changing the down migration to no-op * test: add unit tests for VR dashboard and setup shared test module * fix: fix build errors in DTOs and timeAgo helper * fix ESlint error * changing file names * fix comments * test: implement Object Mother pattern for VerificationRequest mocks * refactor: optimize history module architecture and fix pagination history.controller.ts: - Implemented Thin Controller pattern, delegating orchestration to the service layer. history.service.ts: - Refactored getHistoryParams to support M2M objects, Chatbot strings, and User ObjectIds. - Migrated from .find() to Aggregation Pipeline for complex polymorphic user lookups. - Fixed pagination bug to ensure correct data loading for multiple pages. history.interfaces.ts: - Centralized Query, Response, and Props interfaces historyApi.ts: - Improved nomenclature and added error handling. - Refined type safety for optional parameters. LocalizedDate.tsx: - Improved type safety and added defensive checks for invalid dates. - Removed hydration workarounds in favor of a native i18n implementation. HistoryListItem.tsx: - Applied Static Extraction pattern to display logic for better memory efficiency. - Implemented null-safety handling to prevent UI crashes with malformed data. - Added useMemo for memoization of display names to optimize re-renders. * finishing some types and correcting sonar cloud comments * feat: add migration to convert string user IDs to ObjectId in history collection * refactor: move stats logic to dedicated service and update unit test architecture * test: add base spec with manual provider mocking for verification-request.service.ts * removing logic from the verification request service * correct name and import correctly * adding specific typing for users and improving logic for saving internal users * add migration flag to revert only changed users * adding a lookup conditional that runs only when user is objectid * adding type and conditional safety to getDescriptionForHide * fix comments: extract verification request mocks and add error handling to stats * feat: zod implementation and endpoint test * test: add unit tests for image claim review endpoints * refactor: improve types and add validation for targetId and targetModel * test: add unit tests for history service and controller with mocks * fix: update chatbot user payload to use clientId and add frontend type guard * feat(migration): convert history user strings to M2M objects * fix imports * test: add unit tests and improve data_hash validation * removing IPersonalityService to stop breaking CI * chore: bump node version * Validate data_hash with Zod in image service * fix getByDataHashDto import * Fix: using flex display and centering the banner * Fix lint issues in claim controller tests * fix(home-feed): fix HomeFeedList rendering with Grid container * feat: recaptcha verification on user creation * feat: sign up cypress tests * fix: code smells * fix: simplified test coverage and fix tests checks * fix: cypress tests and tsconfig update * feat: improve claim mocks * chore: optimize test suite performance and fix errors Implemented comprehensive test optimization to improve performance and reliability: **Performance Improvements:** - Reduced total test execution time from ~197s to ~126s (36% faster) - All 70 tests now passing with 100% success rate - Eliminated timeout and duplicate key errors **Key Changes:** 1. Shared MongoDB Instance - Created global setup/teardown for single MongoMemoryServer instance - Eliminated per-test-suite MongoDB instances - Reduced startup overhead significantly 2. Database Cleanup - Added CleanupDatabase utility to clear collections between test suites - Prevents duplicate key errors from hardcoded test data IDs - Ensures clean state for each test suite 3. Test Configuration - Set maxWorkers to 1 for sequential execution (avoids data conflicts) - Increased test timeout to 30s for reliability - Configured MongoDB binary caching 4. Updated Test Files - Removed individual MongoMemoryServer instances - Added database cleanup in afterAll hooks - Updated all e2e tests to use shared MongoDB instance Files modified: - server/tests/globalSetup.ts (new) - server/tests/globalTeardown.ts (new) - server/tests/utils/CleanupDatabase.ts (new) - server/tests/jest-e2e.config.json - jest-mongodb-config.js (new) - All e2e test files and parser.service.spec.ts * perf: enable parallel test execution with upsert operations Achieved 67% faster test execution by enabling parallel test runs. **Performance Results:** - Original (with timeouts): ~197s - Sequential optimization: 125.687s (36% improvement) - Parallel execution: 65.429s (67% improvement, 48% faster than sequential) **Key Changes:** 1. Enabled parallel execution (maxWorkers: 50% in jest-e2e.config.json) 2. Updated SeedTestUser to use upsert operation instead of insertOne 3. Updated SeedTestPersonality to use bulkWrite with upsert operations 4. Eliminated duplicate key errors during parallel test execution **Technical Details:** - Changed seed functions from insertOne/insertMany to updateOne/bulkWrite with upsert: true - This makes seed operations idempotent and safe for concurrent execution - Tests can now run in parallel without data conflicts - All 70 tests passing with 100% success rate Test Results: - Test Suites: 7 passed, 7 total - Tests: 70 passed, 70 total - Time: 65.429s (vs 125.687s sequential) * chore: improve test suite robustness and maintainability * fix: properly json parsing sitekey * draft: track issues automation * UI: standardize translations and extract tag logic into helpers * code smells * remove comment * feat: enhance code quality * adding fallback in getSeverityColor * unifying M2M types * chore(deps): update dependencies with minimal risk * setting isURLField to true * fix: allow users to remove topics from verification requests * refactor: linked issue graphql * refactor: restored regex to match template and draft PR on in progress * chore(lint): remove unsed eslint-disable directives * chore(lint): migrate to Eslint flat config * fix(ci): migrate ESLint to flat config * fix(ci): migrate ESLint to flat config * final adjustments after rebase * Improving nomenclature * refactor/docs: improve types and add explanatory comments * refactor: consolidating into one job * feat(api): validate verificationRequestId using HEX24 pattern * test(tracking): add unit tests and improve error handling in service and controller * fix(eslint): use react version detection instead of hardcoded value * UI: Adding a classification chip to Kanban: * refactor: add VerificationRequestStatus enum and fix imports * refactor(tracking): implement strategy pattern and decouple business logic * refactor: replace console.log with NestJS Logger and enhance Winston config * feat: Add global exception filter for consistent error handling and logging, add request ID tracking for log correlation across requests, add uncaught exception and unhandled rejection handler * feat: using crypto.randomUUID() intead of mathRandom and WinstonLogger.error() to accept string | Error as the second argument * resolving last comments * option to disable field in dynamic form :wq * refactor: standardize dynamic form for creation and editing * refactor: improve extra sources input and unify verification request form types * fix: Aligment on verification request dashboard itens and pie chart with one source * refactor(namespaces): convert form to DynamicForm and fix slug update logic * refactor: convert badge creation and edition to dynamic form * Code Smell: Impact Area field should be optional * fix: ensure api receives only the params to change * UI: Adding a classification chip to Kanban * UI/Refactor: adding tracking card on verification request page and improving latestStatus const * UX: add feedback when step date is missing * test(tracking): adapt unit tests for currentStatus logic * Fix: Remove trailing paragraph from reports stored in database (#2201) * Fix: Remove trailing paragraph from reports stored in database * Fix: validate fields before unset in migration and clean up VisualEditor styles * Add comment explaining trailing paragraph removal in review task actions * refactor: rolling back migration and removing paragraph on serve side * adding tests * fix: cypress review test * fix: removing problematic cleanedVisualEditor --------- Co-authored-by: caneppelevitor <vitor@amplifymd.com> * solve comments * fix: add typed props, null guard, and design cleanup to KanbanCard classification chip * flex start classification chip * solve comments * chore: update langchain to latest version and typescript to v5 * refactor: replace createOpenAIFunctionsAgent with createToolCallingAgent * refactor(ui): close drawer instead of triggering browser back on cancel * fix: synchronize and sanitize source list to prevent validation errors on empty inputs * feat: allow creation of custom impact area not present in autocomplete suggestions * adding data-cy * test: implement end-to-end tests for the verification request lifecycle * fix cypress error * test: decouple creation and edition flows into separate test cases * test: navigate to verification request page using data hash identifier * fix: resolve ansi-styles ESM conflict for @langchain/core * refactor(source-list): optimize state and add debounced input * fix comments * feat(auth): add global conditional utility for role verification * chore: update nestjs/common from 9.2.0 to 10.4.22 * chore: update @casl/ability from 6.7.3 to 6.8.0 * chore: update ai-sdk from 3.4.33 to 5.0.52 * fix(release): using the corrected decorator * refactor(date): create date utils and replace direct MUI/dayjs usage * fix: racing conditions on wikidata cache and using findOneAndUpdate for creation * fix(release): restore VR board column chips and pagination logic Restores changes from #2124 that were lost during conflict resolution. Fixes: - Total VR count display on column chips - 'Load More' button visibility logic - 'No Results' message conditional rendering * fix: migration to fix nested source arrays from legacy verification requests * fix: regenerate lockfile * feat: fixing nested arrays on topics --------- Co-authored-by: Lucas Lobato Biazotto <106782971+lucaslobatob@users.noreply.github.com> Co-authored-by: Lucas Lobato <lucasbiazotto2009@hotmail.com> Co-authored-by: Eduardo <eduardo@tsqrdinc.com> Co-authored-by: Jamerson Santos (James) <james.santos@aletheiafact.org> Co-authored-by: Eduardo <neves98@outlook.com> Co-authored-by: Vítor Caneppele <99989478+caneppelevitor@users.noreply.github.com> Co-authored-by: caneppelevitor <vitor.a.caneppele@gmail.com> Co-authored-by: thesocialdev <mateusbatistasantos@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: caneppelevitor <vitor@amplifymd.com> Co-authored-by: Enzo Bersi <enzobersi06@gmail.com>

Fix: Remove trailing paragraph from reports stored in database

583f493

Jamerson-santos requested a review from thesocialdev as a code owner January 27, 2026 14:19

github-project-automation bot added this to AletheiaFact Backlog Jan 27, 2026

Jamerson-santos requested review from LuizFNJ, caneppelevitor, lucaslobatob and snowmous3 January 27, 2026 14:20

LuizFNJ reviewed Jan 27, 2026

View reviewed changes

src/components/Collaborative/VisualEditor.style.tsx Outdated Show resolved Hide resolved

LuizFNJ approved these changes Jan 27, 2026

View reviewed changes

caneppelevitor requested changes Jan 28, 2026

View reviewed changes

migrations/20260114110506-remove-trailing-paragraph-from-review-tasks.ts Outdated Show resolved Hide resolved

src/components/Collaborative/VisualEditor.style.tsx Outdated Show resolved Hide resolved

src/machines/reviewTask/actions.ts Outdated Show resolved Hide resolved

Jamerson-santos self-assigned this Jan 28, 2026

Fix: validate fields before unset in migration and clean up VisualEdi…

68e5351

…tor styles

Jamerson-santos requested a review from caneppelevitor January 28, 2026 18:42

Add comment explaining trailing paragraph removal in review task actions

3fd1bf0

thesocialdev reviewed Feb 2, 2026

View reviewed changes

migrations/20260114110506-remove-trailing-paragraph-from-review-tasks.ts Outdated Show resolved Hide resolved

caneppelevitor added 4 commits February 16, 2026 09:10

refactor: rolling back migration and removing paragraph on serve side

8c9bea1

adding tests

fdf6c5c

fix: cypress review test

df6671a

fix: removing problematic cleanedVisualEditor

2df3af3

thesocialdev merged commit 4057a4a into stage Feb 17, 2026
6 of 8 checks passed

github-project-automation bot moved this to QA in AletheiaFact Backlog Feb 17, 2026

LuizFNJ mentioned this pull request Feb 25, 2026

Release/v1.2.0 #2264

Merged

46 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix: Remove trailing paragraph from reports stored in database#2201

Fix: Remove trailing paragraph from reports stored in database#2201
thesocialdev merged 7 commits intostagefrom
fix/remove-trailing-paragraphs

Jamerson-santos commented Jan 27, 2026 •

edited by caneppelevitor

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud bot commented Jan 29, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Jamerson-santos commented Jan 27, 2026 • edited by caneppelevitor Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Testing

Backend / Data validation

Backward compatibility

Editor regression checks (core workflow)

Developer Checklist

General

Frontend Changes

Backend Changes

Tests

Test IDs

Merge Request Review Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud bot commented Jan 29, 2026

Quality Gate passed

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jamerson-santos commented Jan 27, 2026 •

edited by caneppelevitor

Loading