Skip to content

[#13590] Refactor student activity logs to write directly to PostgreSQL instead of parsing logs#13692

Open
PhoebeY05 wants to merge 43 commits intoTEAMMATES:masterfrom
PhoebeY05:refactor-student-activity-logs
Open

[#13590] Refactor student activity logs to write directly to PostgreSQL instead of parsing logs#13692
PhoebeY05 wants to merge 43 commits intoTEAMMATES:masterfrom
PhoebeY05:refactor-student-activity-logs

Conversation

@PhoebeY05
Copy link
Copy Markdown
Contributor

@PhoebeY05 PhoebeY05 commented Mar 30, 2026

Fixes #13590

Outline of Solution

  • Save feedback session logs to SQL synchronously
  • Preserve spam filtering
  • Remove cloud-log backfill dependency
  • Add retention cleanup
  • Update retention and frontend behavior

@github-actions
Copy link
Copy Markdown

Hi @PhoebeY05, thank you for your interest in contributing to TEAMMATES!
However, your PR does not appear to follow our contribution guidelines:

  • Title must start with the issue number the PR is fixing in square brackets, e.g. [#<issue-number>]

Please address the above before we proceed to review your PR.

@PhoebeY05 PhoebeY05 changed the title Save feedback session logs to SQL synchronously [#13590] Refactor student activity logs to write directly to PostgreSQL instead of parsing logs Mar 30, 2026
@PhoebeY05 PhoebeY05 marked this pull request as ready for review April 1, 2026 08:59
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors student activity logging to persist feedback session activity directly into PostgreSQL at the time the activity occurs, removing the previous dependency on periodic cloud-log parsing/backfill and adding a retention cleanup cron job.

Changes:

  • Persist feedback session activity logs synchronously via CreateFeedbackSessionLogAction with spam filtering.
  • Remove the legacy cron ingestion flow (UpdateFeedbackSessionLogsAction) and associated log-service APIs/DTOs.
  • Add a weekly cron endpoint to delete logs older than 90 days and update UI/docs to reflect immediate availability + new retention.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/web/app/pages-instructor/instructor-student-activity-logs/instructor-student-activity-logs.component.ts Updates frontend retention constant usage and removes “last updated” ingestion-delay messaging.
src/web/app/pages-instructor/instructor-student-activity-logs/instructor-student-activity-logs.component.html Updates instructor-facing copy to reflect immediate log availability and 90-day retention.
src/web/app/pages-instructor/instructor-student-activity-logs/snapshots/instructor-student-activity-logs.component.spec.ts.snap Updates snapshots for new retention value and copy.
src/web/app/components/question-types/question-edit-answer-form/constsum-options-question-edit-answer-form.component.html Adjusts Angular @for tracking to use $index.
src/test/java/teammates/sqlui/webapi/UpdateFeedbackSessionLogsActionTest.java Removes tests for deprecated cron ingestion action.
src/test/java/teammates/sqlui/webapi/GetFeedbackSessionLogsActionTest.java Aligns tests with new student-activity retention constant.
src/test/java/teammates/sqlui/webapi/GetActionClassesActionTest.java Updates action class list (remove ingestion action, add cleanup action).
src/test/java/teammates/sqlui/webapi/CreateFeedbackSessionLogActionTest.java Adds verification for DB persistence + spam-filter behavior.
src/test/java/teammates/sqlui/webapi/CleanupFeedbackSessionLogsActionTest.java Adds unit test coverage for new cleanup cron action.
src/test/java/teammates/logic/api/MockLogsProcessor.java Removes feedback-session-log mock behavior tied to the deleted ingestion flow.
src/main/java/teammates/ui/webapi/UpdateFeedbackSessionLogsAction.java Removes legacy cron ingestion action.
src/main/java/teammates/ui/webapi/CreateFeedbackSessionLogAction.java Writes activity logs directly to SQL with spam filtering.
src/main/java/teammates/ui/webapi/CleanupFeedbackSessionLogsAction.java Adds cron endpoint to delete logs older than retention period.
src/main/java/teammates/ui/webapi/ActionFactory.java Remaps cron URI from ingestion to cleanup endpoint.
src/main/java/teammates/ui/constants/ApiConst.java Exposes new student-activity retention constant to frontend.
src/main/java/teammates/storage/sqlapi/FeedbackSessionLogsDb.java Adds “latest log” query + bulk delete-by-cutoff mutation.
src/main/java/teammates/sqllogic/core/FeedbackSessionLogsLogic.java Adds logic-layer passthroughs for latest-log query and deletion.
src/main/java/teammates/sqllogic/api/Logic.java Exposes latest-log query and deletion methods to actions.
src/main/java/teammates/logic/external/LogService.java Removes feedback-session-log methods (now SQL-backed, not logging-service-backed).
src/main/java/teammates/logic/external/LocalLoggingService.java Removes local in-memory feedback-session-log simulation (now SQL-backed).
src/main/java/teammates/logic/external/GoogleCloudLoggingService.java Removes cloud feedback-session-log parsing methods.
src/main/java/teammates/logic/api/LogsProcessor.java Removes feedback-session-log APIs tied to external logging services.
src/main/java/teammates/common/util/Const.java Introduces 90-day student activity retention, removes ingestion interval constant, updates cron URI.
src/main/java/teammates/common/datatransfer/FeedbackSessionLogEntry.java Removes DTO previously used for parsed-log ingestion.
src/it/java/teammates/it/ui/webapi/UpdateFeedbackSessionLogsActionIT.java Removes IT coverage for deprecated ingestion cron action.
src/it/java/teammates/it/ui/webapi/GetFeedbackSessionLogsActionIT.java Aligns IT with new retention constant.
src/it/java/teammates/it/ui/webapi/CreateFeedbackSessionLogActionIT.java Adds IT assertions that logs are persisted and spam-filtered.
src/it/java/teammates/it/ui/webapi/CleanupFeedbackSessionLogsActionIT.java Adds IT coverage for cleanup behavior.
src/it/java/teammates/it/storage/sqlapi/FeedbackSessionLogsDbIT.java Adds IT coverage for latest-log query and cutoff deletion.
src/it/java/teammates/it/sqllogic/core/FeedbackSessionLogsLogicIT.java Adds IT coverage for logic-layer latest-log query and cutoff deletion.
docs/cron-setup.md Updates cron documentation to replace ingestion schedule with weekly cleanup job + new retention details.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

@DhiraPT DhiraPT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Please note the comments.

@PhoebeY05 PhoebeY05 requested a review from DhiraPT April 4, 2026 07:45
@samuelfangjw
Copy link
Copy Markdown
Member

@PhoebeY05 need to course correct a bit, please remove the deduplication logic. The added complexity coming from deduplication, handling lock contention and reasoning about race conditions is not worth it.

Ensuring the endpoint is not publicly accessible without account or regkey is enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor student activity logs to write directly to PostgreSQL instead of parsing logs

4 participants