You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.rst
+13-7Lines changed: 13 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,10 +25,16 @@ Scrapyrt (Scrapy realtime)
25
25
HTTP server which provides API for scheduling Scrapy spiders and
26
26
making requests with spiders.
27
27
28
-
Allows you to easily add HTTP API to your existing Scrapy project. All Scrapy project
29
-
components (e.g. middleware, pipelines, extensions) are supported out of the box. You
30
-
simply run Scrapyrt in Scrapy project directory and it starts HTTP server allowing you
31
-
to schedule your spiders and get spider output in JSON format.
28
+
Features
29
+
========
30
+
* Allows you to easily add HTTP API to your existing Scrapy project
31
+
* All Scrapy project components (e.g. middleware, pipelines, extensions) are supported out of the box.
32
+
* You simply run Scrapyrt in Scrapy project directory and it starts HTTP server allowing you to schedule your spiders and get spider output in JSON format.
33
+
34
+
Note
35
+
====
36
+
* Project is not a replacement for `Scrapyd <https://scrapyd.readthedocs.io/en/stable/>`_ or `Scrapy Cloud <https://www.zyte.com/scrapy-cloud/>`_ or other infrastructure to run long running crawls
37
+
* Not suitable for long running spiders, good for spiders that will fetch one response from some website and return response
32
38
33
39
Getting started
34
40
===============
@@ -66,7 +72,7 @@ with callback ``parse_foo``::
66
72
Documentation
67
73
=============
68
74
69
-
Documentation is available here: http://scrapyrt.readthedocs.org/en/latest/index.html
75
+
`Documentation is available on readthedocs <http://scrapyrt.readthedocs.org/en/latest/index.html>`_
70
76
71
77
Support
72
78
=======
@@ -81,9 +87,9 @@ Commercial support is also available by `Scrapinghub`_.
81
87
82
88
License
83
89
=======
84
-
ScrapyRT is offered under [BSD 3Clause license](https://en.wikipedia.org/wiki/BSD_licenses).
90
+
ScrapyRT is offered under `BSD 3-Clause license<https://en.wikipedia.org/wiki/BSD_licenses#3-clause_license_(%22BSD_License_2.0%22,_%22Revised_BSD_License%22,_%22New_BSD_License%22,_or_%22Modified_BSD_License%22)>`_.
85
91
86
92
87
93
Development
88
94
===========
89
-
Development taking place on github: https://github.com/scrapinghub/scrapyrt
95
+
Development taking place on `Github <https://github.com/scrapinghub/scrapyrt>`_.
0 commit comments