Skip to content

Conversation

@a-klos
Copy link
Member

@a-klos a-klos commented Jan 7, 2026

This pull request adds support for a new continue_on_failure option to sitemap-based document uploads, allowing users to specify whether the crawl should skip pages that fail to load rather than aborting the entire process. The feature is integrated across the backend, frontend, and documentation, and includes robust handling and testing for various input types.

Backend changes:

  • Added parsing and normalization for the continue_on_failure parameter in _parse_sitemap_loader_parameters, supporting boolean and string representations.
  • Defaulted continue_on_failure to True in aextract_content if not provided, ensuring failure resilience by default.

Frontend changes:

  • Added a checkbox for "Continue on failure" to the sitemap upload UI (DocumentUploadContainer.vue), defaulting to checked [1] [2].
  • Passed the continueOnFailure value from the UI through the API payload [1] [2].
  • Updated the SitemapConfig interface to include continueOnFailure.

Documentation and localization:

  • Documented the new parameter in README.md for sitemap sources.
  • Added English and German translations for the new UI option and its tooltip [1] [2].

Testing:

  • Updated and expanded tests to verify correct handling and passing of the continue_on_failure parameter, including type normalization and defaulting behavior [1] [2] [3] [4].

@a-klos
Copy link
Member Author

a-klos commented Jan 7, 2026

fixes #201

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants