feat(html): add default_content_layer option to HTMLBackendOptions#3098
Open
M-Hassan-Raza wants to merge 3 commits intodocling-project:mainfrom
Open
feat(html): add default_content_layer option to HTMLBackendOptions#3098M-Hassan-Raza wants to merge 3 commits intodocling-project:mainfrom
M-Hassan-Raza wants to merge 3 commits intodocling-project:mainfrom
Conversation
Add a `default_content_layer` field to `HTMLBackendOptions` that allows users to explicitly set the starting content layer, bypassing the automatic inference based on `infer_furniture` and header presence. This is useful for HTML content from API endpoints that may not contain headings but should have all content in the BODY layer. Closes docling-project#2487 Signed-off-by: Hassan Raza <raihassanraza10@gmail.com>
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
|
Related Documentation 1 document(s) may need updating based on files changed in this PR: Docling Content LayersView Suggested Changes@@ -94,6 +94,42 @@
Currently, the ability to include furniture in exports is only available via the Python API. The `docling-serve` API and CLI exports do not support specifying content layers and will always export with the default (BODY only).
+## HTML Backend Content Layer Options
+
+When processing HTML documents, Docling provides fine-grained control over content layer assignment through `HTMLBackendOptions`. The `default_content_layer` parameter allows you to explicitly set the starting content layer for HTML documents, overriding the automatic inference behavior.
+
+### Automatic Content Layer Inference
+
+By default, when `default_content_layer` is `None`, Docling determines the starting content layer based on:
+1. The `infer_furniture` setting (default: `True`)
+2. The presence of headers in the HTML document
+
+If `infer_furniture` is enabled and headers are present, content before the first header is assigned to the `FURNITURE` layer. Otherwise, all content starts in the `BODY` layer.
+
+### Explicit Content Layer Override
+
+The `default_content_layer` parameter allows you to bypass this inference logic entirely. When set to a specific `ContentLayer` value (e.g., `ContentLayer.BODY`), all content starts in that layer regardless of heading presence or the `infer_furniture` setting.
+
+This is particularly useful for HTML content from API endpoints that may lack headings but should have all content in the `BODY` layer.
+
+**Example:**
+
+```python
+from docling import DocumentConverter
+from docling.datamodel.backend_options import HTMLBackendOptions
+from docling_core.types.doc.document import ContentLayer
+
+# Force all HTML content to start in the BODY layer
+options = HTMLBackendOptions(default_content_layer=ContentLayer.BODY)
+converter = DocumentConverter()
+result = converter.convert(source, options=options)
+```
+
+**Configuration summary:**
+- `default_content_layer=None` (default): Starting layer is determined by `infer_furniture` and header presence
+- `default_content_layer=ContentLayer.BODY`: All content starts in the BODY layer
+- `default_content_layer=ContentLayer.FURNITURE`: All content starts in the FURNITURE layer
+
## Customization and Post-processing
Headers and footers are detected automatically by Docling’s layout model for `.docx` files. There is currently no rule-based mechanism to customize their detection during processing. However, you can manually remove or further process these elements after extraction if needed.Note: You must be authenticated to accept/decline updates. |
Add a negative test verifying that without default_content_layer override, pre-heading content is still correctly inferred as FURNITURE. Extract shared HTML fixture and helper function. Signed-off-by: Hassan Raza <raihassanraza10@gmail.com>
Contributor
|
✅ DCO Check Passed Thanks @M-Hassan-Raza, all your commits are properly signed off. 🎉 |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Signed-off-by: Hassan Raza <raihassanraza10@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
default_content_layeroption toHTMLBackendOptionsthat allows users to explicitly set the starting content layer, bypassing the automatic inference based oninfer_furnitureand header presenceNone(default), the existing behavior is preservedContentLayer.BODY), all content starts in that layer regardless of heading presenceThis is useful for HTML content from API endpoints that may lack headings but should have all content in the
BODYlayer.Closes #2487
Test plan
pre-commit run --all-filespasses (Ruff, MyPy, uv-lock)pytest tests/test_backend_html.py— all 20 tests pass (including newtest_html_backend_default_content_layer)default_content_layer=ContentLayer.BODY, content before a heading stays in BODY instead of being inferred as FURNITURE