sphinx-scrapy

Sphinx extension for documentation in the Scrapy ecosystem.

Features

• Automatic configuration of intersphinx for Python and Scrapy.

  Ready-to-use, easy-to-enable configuration for the following packages of the Scrapy ecosystem is also available:

  attrs

  cssselect

  dateparser

  form2request

  formasaurus

  itemloaders

  jinja

  lxml

  packaging

  parsel

  pydantic

  pytest

  python-scrapinghub

  python-zyte-api

  scrapy-poet

  scrapy-spider-metadata

  scrapy-zyte-api

  scrapy-zyte-smartproxy

  scrapyd

  shub

  shub-image

  spidermon

  tenacity

  twisted (and twistedapi)

  url-matcher

  w3lib

  web-poet

  zyte

  zyte-common-items

  zyte-parsers

  zyte-spider-templates

  To automatically configure intersphinx for any of those packages if installed, add to your conf.py file:

  scrapy_intersphinx_enable_installed = True

  You can also enable or disable the automatic intersphinx configuration of packages manually:

  scrapy_intersphinx_enable = [
      "parsel",
      "w3lib",
  ]
  scrapy_intersphinx_disable = [
      "scrapy",
  ]

• Automatic configuration of Sphinx roles of the Scrapy documentation, so that you can easily link to Scrapy settings, request metadata keys, signals and commands:

  :setting:`BOT_NAME`
  :setting:`LOG_LEVEL <scrapy:LOG_LEVEL>`
  :reqmeta:`download_slot`
  :signal:`spider_opened`
  :command:`crawl`

Setup

1. Install:

   pip install sphinx-scrapy

2. Add to your conf.py:

   extensions = [
       "sphinx.ext.intersphinx",
       "sphinx_scrapy",
   ]

Release notes

See Release notes for a list of releases and their changes.