Skip to content

Conversation

@thongdk8
Copy link
Contributor

@thongdk8 thongdk8 commented Nov 18, 2025

Description

This PR addresses the issue of unexpected exceptions (non-TransactionException) are not properly caught when importing data in transaction mode with misconfigured transaction managers, causing the import process to fail ungracefully.

For example, in the STORAGE mode of importing:

  • Using single-crud-operation transaction manager throws UnsupportedOperationException when starting a transaction
  • Start transaction while connecting to a cluster configured with single-crud-operation results in grpc.StatusRuntimeException: UNIMPLEMENTED on the client side (using scalardb-cluster-data-loader)

These exceptions bypass the existing TransactionException handler, leaving transactions unaborted and errors unlogged with proper context.

Related issues and/or PRs

N/A

Changes made

  • Added a generic catch (Exception e) block in ImportProcessor.processTransactionBatch() to handle unexpected exceptions
  • Enhanced error logging to include batch ID and data chunk ID for better debugging
  • Ensured transactions are properly aborted even when unexpected exceptions occur
  • Error messages now include exception class name and message in format: "Unexpected error: <ExceptionClass> - <message>"
  • Added unit test process_withUnexpectedExceptionInTransaction_shouldHandleGracefully() to verify the new exception handling

Checklist

  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes.
  • I have considered whether similar issues could occur in other products, components, or modules if this PR is for bug fixes.
  • Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
  • Tests (unit, integration, etc.) have been added for the changes.
  • My changes generate no new warnings.
  • Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

NA

Release notes

Fixed data-loader to properly handle unexpected exceptions during transaction processing for importing, ensuring transactions are aborted and errors are logged with proper context.

@thongdk8 thongdk8 self-assigned this Nov 18, 2025
@thongdk8
Copy link
Contributor Author

/gemini review

@thongdk8 thongdk8 requested a review from Copilot November 18, 2025 09:26
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively addresses a critical issue in the data loader's exception handling by catching unexpected exceptions during transaction processing, ensuring transactions are properly aborted, and improving error logging. The changes are logical and the inclusion of a unit test to verify the new behavior is commendable. I have a few suggestions to further enhance the code's robustness and maintainability.

Copilot finished reviewing on behalf of thongdk8 November 18, 2025 09:35
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances exception handling in the data loader's transaction processing to properly catch and handle unexpected exceptions (non-TransactionException) that can occur with misconfigured transaction managers. The changes ensure transactions are properly aborted and errors are logged with appropriate context even when unexpected exceptions occur.

Key Changes:

  • Added a generic catch (Exception e) block to handle unexpected exceptions during transaction processing
  • Enhanced error logging to include batch ID and data chunk ID for better debugging context
  • Added unit test to verify graceful handling of unexpected exceptions

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
data-loader/core/src/main/java/com/scalar/db/dataloader/core/dataimport/processor/ImportProcessor.java Added generic exception handler with enhanced logging and proper transaction abort logic for unexpected exceptions
data-loader/core/src/test/java/com/scalar/db/dataloader/core/dataimport/processor/ImportProcessorTest.java Added unit test process_withUnexpectedExceptionInTransaction_shouldHandleGracefully() to verify proper handling of unexpected exceptions in transaction mode

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@thongdk8 thongdk8 requested review from a team, Torch3333, brfrn169, feeblefakie, komamitsu and ypeckstadt and removed request for a team November 18, 2025 09:45
@thongdk8 thongdk8 marked this pull request as ready for review November 18, 2025 09:46
@thongdk8 thongdk8 requested a review from inv-jishnu November 18, 2025 09:53
Copy link
Collaborator

@brfrn169 brfrn169 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

Copy link
Contributor

@ypeckstadt ypeckstadt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you.

Copy link
Contributor

@inv-jishnu inv-jishnu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Thank you!

Copy link
Contributor

@Torch3333 Torch3333 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@thongdk8 thongdk8 requested a review from komamitsu November 19, 2025 07:32
Copy link
Contributor

@komamitsu komamitsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@thongdk8 thongdk8 requested a review from feeblefakie November 19, 2025 10:28
Copy link
Contributor

@feeblefakie feeblefakie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@brfrn169 brfrn169 enabled auto-merge (squash) November 20, 2025 01:42
@brfrn169 brfrn169 merged commit fd37bca into master Nov 20, 2025
71 of 72 checks passed
@brfrn169 brfrn169 deleted the fix/data-loader/handle-unkown-exception-when-importing branch November 20, 2025 01:48
feeblefakie pushed a commit that referenced this pull request Nov 20, 2025
feeblefakie pushed a commit that referenced this pull request Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants