Skip to content

skip document during indexing if error occurs#11

Open
tonysun83 wants to merge 1 commit intomasterfrom
skip-doc-if-error
Open

skip document during indexing if error occurs#11
tonysun83 wants to merge 1 commit intomasterfrom
skip-doc-if-error

Conversation

@tonysun83
Copy link
Contributor

When an error is returned by search3-java, we handle it by throwing a
generic bad_request exception. This is the error handling path for both
search and indexing requests. During indexing however, the exception is
not caught and it leads to a search_worker_manager process crash. The
process is restarted by the search3_sup supervisor and the indexing
begins again. This leads to an endless loop of trying to index the same
document if the documents causes some sort of error. We modify this
behavior by catching the thrown exception and ignoring the document.

When an error is returned by search3-java, we handle it by throwing a
generic bad_request exception. This is the error handling path for both
search and indexing requests. During indexing however, the exception is
not caught and it leads to a search_worker_manager process crash. The
process is restarted by the search3_sup supervisor and the indexing
begins again. This leads to an endless loop of trying to index the same
document if the documents causes some sort of error. We modify this
behavior by catching the thrown exception and ignoring the document.
@tonysun83 tonysun83 requested a review from rnewson November 4, 2019 06:35
@tonysun83 tonysun83 changed the title skip error during indexing skip document during indexing if error occurs Nov 4, 2019
Copy link
Contributor

@rnewson rnewson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidenote: verify_same_session is more idiomatically written as

% Session Verification
verify_same_session(Session, Session) when is_binary(Session) ->
    Session;
verify_same_session(_, _) ->
    throw(session_mismatch).


handle_update_response({ok, #{session := Session}, _}, CurrentSession) ->
VSession = verify_same_session(CurrentSession, Session),
VSession;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't look like the right result. handle_response returns {VSession, Response} above, for example.

Copy link
Contributor Author

@tonysun83 tonysun83 Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rnewson handle_response was a generic handler and Response was need for other rpc requests. However in the case of updates, Response isn't needed, so I removed it.

@rnewson rnewson self-requested a review November 4, 2019 23:38
Copy link
Contributor

@rnewson rnewson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants