Skip to content

Harmony 1963 - data size change information in job status page#710

Merged
indiejames merged 10 commits intomainfrom
harmony-1963
Mar 10, 2025
Merged

Harmony 1963 - data size change information in job status page#710
indiejames merged 10 commits intomainfrom
harmony-1963

Conversation

@indiejames
Copy link
Copy Markdown
Contributor

Jira Issue ID

HARMONY-1963

Description

Adds information about data size change from original to output

Local Test Steps

Run a few queries and look at the job status pages. Verify that the size information is at the bottom. For example

"originalDataSize": "1.37 MiB",
"outputDataSize": "92.53 KiB",
"dataSizePercentChange": "93.41% reduction"

Make sure the size change looks right.

PR Acceptance Checklist

  • Acceptance criteria met
  • Tests added/updated (if needed) and passing
  • Documentation updated (if needed)
  • Harmony in a Box tested (if changes made to microservices or new dependencies added)

} else if (diff > 0) {
let percent = (diff / sizes.originalSize * 100.0).toFixed(precision);
// due to JS precision issues, big changes will appear to be 100% reduction, which is impossible
if (percent === '100.00') percent = '99.99';
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This will not work if the precision is not the default (2).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed and test added.

@indiejames indiejames requested a review from ygliuvt March 6, 2025 17:47
Copy link
Copy Markdown
Member

@ygliuvt ygliuvt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested successfully locally.

Copy link
Copy Markdown
Contributor

@chris-durbin chris-durbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial reaction is that I think we need to store these values and calculate them once.

  1. Work items are transient (they get cleaned up after a job completes based on a configuration interval).
  2. Calculating it on every call to get the job status for a completed request could be slow for something with a lot of granules.

hookTransaction();
before(async function () {
await aJob.save(this.trx);
console.log(`JOB ID: ${aJob.jobID}`);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@indiejames
Copy link
Copy Markdown
Contributor Author

My initial reaction is that I think we need to store these values and calculate them once.

  1. Work items are transient (they get cleaned up after a job completes based on a configuration interval).
  2. Calculating it on every call to get the job status for a completed request could be slow for something with a lot of granules.

Done

@indiejames indiejames requested a review from chris-durbin March 10, 2025 13:44
Copy link
Copy Markdown
Contributor

@chris-durbin chris-durbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested successfully locally.

@indiejames indiejames merged commit 1a63f2b into main Mar 10, 2025
5 checks passed
@indiejames indiejames deleted the harmony-1963 branch August 18, 2025 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants