Skip to content

Conversation

@nbbeeken
Copy link
Contributor

@nbbeeken nbbeeken commented Nov 4, 2024

Description

What is changing?

  • Take data out of ram and write it to file
  • process file externally to find median
  • Skip ldjsonMultiFileUpload
Is there new documentation needed for these changes?

No

What is the motivation for this change?

ldjsonMultiFileUpload is failing
returnDocument produces more data points than a JS array can hold

Double check the following

  • Ran npm run check:lint script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@nbbeeken nbbeeken marked this pull request as ready for review November 4, 2024 22:26

const parallelBench = average([
microBench.parallel.ldjsonMultiFileUpload,
microBench.parallel.ldjsonMultiFileUpload ?? 0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worry that setting it to 0 would mess up the tracked result, could we just take it out? Or set it to the currently known average value if we're concerned about preserving the parallel measurement history?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, removing it makes more sense. I'd rather have accurate numbers than simulate a value here if that sounds good.

@nbbeeken nbbeeken requested a review from dariakp November 4, 2024 23:21
@nbbeeken nbbeeken changed the title chore: skip ldjsonMultiFileUpload and use file to store data chore: use file to store data in benchmark Nov 5, 2024
@nbbeeken nbbeeken closed this Nov 5, 2024
@nbbeeken nbbeeken deleted the chore-fix-bench branch November 5, 2024 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants