Skip to content

Conversation

@e11sy
Copy link
Contributor

@e11sy e11sy commented Oct 10, 2025

CPU usage reduced:

  • do not store ~380k records in one object
  • do not cast .toArray() the whole db call result
  • do not iterate over the large object several times (only one time really required)

Db bottleneck solution:

  • add projection to reduce size of the package sent from db to the api

@neSpecc neSpecc requested a review from Copilot October 10, 2025 22:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes chart data processing by reducing CPU usage and database bottlenecks when handling large datasets (~380k records). The changes eliminate unnecessary memory allocation and reduce database payload size.

  • Replaced .toArray() with cursor-based streaming processing to avoid loading all records into memory
  • Added MongoDB projection to fetch only required fields (lastRepetitionTime, groupingTimestamp, count)
  • Implemented direct aggregation during iteration instead of multiple passes over the data

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/models/eventsFactory.js Optimized chart data processing with cursor streaming and direct aggregation
package.json Version bump from 1.2.0 to 1.2.1

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@e11sy e11sy merged commit bc7bf78 into master Oct 10, 2025
6 checks passed
@e11sy e11sy deleted the imp-chart-data-request branch October 10, 2025 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants