[#13590] Refactor student activity logs to write directly to PostgreSQL instead of parsing logs#13692
[#13590] Refactor student activity logs to write directly to PostgreSQL instead of parsing logs#13692PhoebeY05 wants to merge 43 commits intoTEAMMATES:masterfrom
Conversation
|
Hi @PhoebeY05, thank you for your interest in contributing to TEAMMATES!
Please address the above before we proceed to review your PR. |
…/teammates into refactor-student-activity-logs
There was a problem hiding this comment.
Pull request overview
Refactors student activity logging to persist feedback session activity directly into PostgreSQL at the time the activity occurs, removing the previous dependency on periodic cloud-log parsing/backfill and adding a retention cleanup cron job.
Changes:
- Persist feedback session activity logs synchronously via
CreateFeedbackSessionLogActionwith spam filtering. - Remove the legacy cron ingestion flow (
UpdateFeedbackSessionLogsAction) and associated log-service APIs/DTOs. - Add a weekly cron endpoint to delete logs older than 90 days and update UI/docs to reflect immediate availability + new retention.
Reviewed changes
Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/web/app/pages-instructor/instructor-student-activity-logs/instructor-student-activity-logs.component.ts | Updates frontend retention constant usage and removes “last updated” ingestion-delay messaging. |
| src/web/app/pages-instructor/instructor-student-activity-logs/instructor-student-activity-logs.component.html | Updates instructor-facing copy to reflect immediate log availability and 90-day retention. |
| src/web/app/pages-instructor/instructor-student-activity-logs/snapshots/instructor-student-activity-logs.component.spec.ts.snap | Updates snapshots for new retention value and copy. |
| src/web/app/components/question-types/question-edit-answer-form/constsum-options-question-edit-answer-form.component.html | Adjusts Angular @for tracking to use $index. |
| src/test/java/teammates/sqlui/webapi/UpdateFeedbackSessionLogsActionTest.java | Removes tests for deprecated cron ingestion action. |
| src/test/java/teammates/sqlui/webapi/GetFeedbackSessionLogsActionTest.java | Aligns tests with new student-activity retention constant. |
| src/test/java/teammates/sqlui/webapi/GetActionClassesActionTest.java | Updates action class list (remove ingestion action, add cleanup action). |
| src/test/java/teammates/sqlui/webapi/CreateFeedbackSessionLogActionTest.java | Adds verification for DB persistence + spam-filter behavior. |
| src/test/java/teammates/sqlui/webapi/CleanupFeedbackSessionLogsActionTest.java | Adds unit test coverage for new cleanup cron action. |
| src/test/java/teammates/logic/api/MockLogsProcessor.java | Removes feedback-session-log mock behavior tied to the deleted ingestion flow. |
| src/main/java/teammates/ui/webapi/UpdateFeedbackSessionLogsAction.java | Removes legacy cron ingestion action. |
| src/main/java/teammates/ui/webapi/CreateFeedbackSessionLogAction.java | Writes activity logs directly to SQL with spam filtering. |
| src/main/java/teammates/ui/webapi/CleanupFeedbackSessionLogsAction.java | Adds cron endpoint to delete logs older than retention period. |
| src/main/java/teammates/ui/webapi/ActionFactory.java | Remaps cron URI from ingestion to cleanup endpoint. |
| src/main/java/teammates/ui/constants/ApiConst.java | Exposes new student-activity retention constant to frontend. |
| src/main/java/teammates/storage/sqlapi/FeedbackSessionLogsDb.java | Adds “latest log” query + bulk delete-by-cutoff mutation. |
| src/main/java/teammates/sqllogic/core/FeedbackSessionLogsLogic.java | Adds logic-layer passthroughs for latest-log query and deletion. |
| src/main/java/teammates/sqllogic/api/Logic.java | Exposes latest-log query and deletion methods to actions. |
| src/main/java/teammates/logic/external/LogService.java | Removes feedback-session-log methods (now SQL-backed, not logging-service-backed). |
| src/main/java/teammates/logic/external/LocalLoggingService.java | Removes local in-memory feedback-session-log simulation (now SQL-backed). |
| src/main/java/teammates/logic/external/GoogleCloudLoggingService.java | Removes cloud feedback-session-log parsing methods. |
| src/main/java/teammates/logic/api/LogsProcessor.java | Removes feedback-session-log APIs tied to external logging services. |
| src/main/java/teammates/common/util/Const.java | Introduces 90-day student activity retention, removes ingestion interval constant, updates cron URI. |
| src/main/java/teammates/common/datatransfer/FeedbackSessionLogEntry.java | Removes DTO previously used for parsed-log ingestion. |
| src/it/java/teammates/it/ui/webapi/UpdateFeedbackSessionLogsActionIT.java | Removes IT coverage for deprecated ingestion cron action. |
| src/it/java/teammates/it/ui/webapi/GetFeedbackSessionLogsActionIT.java | Aligns IT with new retention constant. |
| src/it/java/teammates/it/ui/webapi/CreateFeedbackSessionLogActionIT.java | Adds IT assertions that logs are persisted and spam-filtered. |
| src/it/java/teammates/it/ui/webapi/CleanupFeedbackSessionLogsActionIT.java | Adds IT coverage for cleanup behavior. |
| src/it/java/teammates/it/storage/sqlapi/FeedbackSessionLogsDbIT.java | Adds IT coverage for latest-log query and cutoff deletion. |
| src/it/java/teammates/it/sqllogic/core/FeedbackSessionLogsLogicIT.java | Adds IT coverage for logic-layer latest-log query and cutoff deletion. |
| docs/cron-setup.md | Updates cron documentation to replace ingestion schedule with weekly cleanup job + new retention details. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/main/java/teammates/ui/webapi/CleanupFeedbackSessionLogsAction.java
Outdated
Show resolved
Hide resolved
src/it/java/teammates/it/ui/webapi/CleanupFeedbackSessionLogsActionIT.java
Outdated
Show resolved
Hide resolved
src/it/java/teammates/it/ui/webapi/CleanupFeedbackSessionLogsActionIT.java
Outdated
Show resolved
Hide resolved
DhiraPT
left a comment
There was a problem hiding this comment.
Thanks for the PR. Please note the comments.
src/main/java/teammates/ui/webapi/CreateFeedbackSessionLogAction.java
Outdated
Show resolved
Hide resolved
src/it/java/teammates/it/ui/webapi/CleanupFeedbackSessionLogsActionIT.java
Show resolved
Hide resolved
src/it/java/teammates/it/ui/webapi/CleanupFeedbackSessionLogsActionIT.java
Show resolved
Hide resolved
src/test/java/teammates/sqlui/webapi/CreateFeedbackSessionLogActionTest.java
Show resolved
Hide resolved
src/it/java/teammates/it/ui/webapi/CreateFeedbackSessionLogActionIT.java
Show resolved
Hide resolved
…/teammates into refactor-student-activity-logs
|
@PhoebeY05 need to course correct a bit, please remove the deduplication logic. The added complexity coming from deduplication, handling lock contention and reasoning about race conditions is not worth it. Ensuring the endpoint is not publicly accessible without account or regkey is enough. |
Fixes #13590
Outline of Solution