feat: add continue_on_failure option to SitemapExtractor and admin frontend #200
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds support for a new
continue_on_failureoption to sitemap-based document uploads, allowing users to specify whether the crawl should skip pages that fail to load rather than aborting the entire process. The feature is integrated across the backend, frontend, and documentation, and includes robust handling and testing for various input types.Backend changes:
continue_on_failureparameter in_parse_sitemap_loader_parameters, supporting boolean and string representations.continue_on_failuretoTrueinaextract_contentif not provided, ensuring failure resilience by default.Frontend changes:
DocumentUploadContainer.vue), defaulting to checked [1] [2].continueOnFailurevalue from the UI through the API payload [1] [2].SitemapConfiginterface to includecontinueOnFailure.Documentation and localization:
README.mdfor sitemap sources.Testing:
continue_on_failureparameter, including type normalization and defaulting behavior [1] [2] [3] [4].