Skip to content

Commit 7fe669e

Browse files
committed
integrate to HTTP crawlers guide
1 parent 5844f4b commit 7fe669e

13 files changed

+95
-121
lines changed

docs/guides/code_examples/crawler_custom_parser/__init__.py renamed to docs/guides/code_examples/http_crawlers/__init__.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/lexbor_parser.py renamed to docs/guides/code_examples/http_crawlers/lexbor_parser.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/lxml_parser.py renamed to docs/guides/code_examples/http_crawlers/lxml_parser.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/lxml_saxonche_parser.py renamed to docs/guides/code_examples/http_crawlers/lxml_saxonche_parser.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/pyquery_parser.py renamed to docs/guides/code_examples/http_crawlers/pyquery_parser.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/scrapling_parser.py renamed to docs/guides/code_examples/http_crawlers/scrapling_parser.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/selectolax_adaptive_run.py renamed to docs/guides/code_examples/http_crawlers/selectolax_adaptive_run.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111

1212

1313
async def main() -> None:
14-
crawler = AdaptivePlaywrightCrawler(
14+
crawler: AdaptivePlaywrightCrawler = AdaptivePlaywrightCrawler(
1515
max_requests_per_crawl=10,
1616
# Use custom Selectolax parser for static content parsing.
1717
static_parser=SelectolaxLexborParser(),

docs/guides/code_examples/crawler_custom_parser/selectolax_context.py renamed to docs/guides/code_examples/http_crawlers/selectolax_context.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@ class SelectolaxLexborContext(ParsedHttpCrawlingContext[LexborHTMLParser]):
1414
context methods (push_data, enqueue_links, etc.) plus custom helpers.
1515
"""
1616

17-
# It is only for convenience and not strictly necessary, as the
18-
# parsed_content field is already available from the base class.
1917
@property
2018
def parser(self) -> LexborHTMLParser:
2119
"""Convenient alias for accessing the parsed document."""

docs/guides/code_examples/crawler_custom_parser/selectolax_crawler.py renamed to docs/guides/code_examples/http_crawlers/selectolax_crawler.py

File renamed without changes.

docs/guides/code_examples/crawler_custom_parser/selectolax_crawler_run.py renamed to docs/guides/code_examples/http_crawlers/selectolax_crawler_run.py

File renamed without changes.

0 commit comments

Comments
 (0)