You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Yes -- VulnerableCode is a work in progress! Please stay in touch on our `Gitter channel <https://gitter.im/aboutcode-org/vulnerablecode>`_; and if you have any feedback, feel free to `enter an issue in our GitHub repo <https://github.com/nexB/vulnerablecode/issues>`_.
68
67
69
-
VulnerableCode is a work in progress project and will likely go through major changes. Please stay in touch on our `Gitter channel <https://gitter.im/aboutcode-org/vulnerablecode>`_
Copy file name to clipboardExpand all lines: docs/source/tutorial_add_new_importer.rst
+26-27Lines changed: 26 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,9 +3,8 @@
3
3
Add a new importer
4
4
====================
5
5
6
-
This tutorial contains all the things one should know to quickly
7
-
implement an importer.
8
-
A lot of internal sausage about importers could be found inside the
6
+
This tutorial contains all the things one should know to quickly implement an importer.
7
+
Many internal details about importers can be found inside the
9
8
:file:`vulnerabilites/importer.py` file.
10
9
Make sure to go through :ref:`importer-overview` before you begin writing one.
11
10
@@ -15,7 +14,7 @@ TL;DR
15
14
#. Create a new :file:`vulnerabilities/importers/{importer_name.py}` file.
16
15
#. Create a new importer subclass inheriting from the ``Importer`` superclass defined in
17
16
``vulnerabilites.importer``. It is conventional to end an importer name with *Importer*.
18
-
#. Specify the importer licence.
17
+
#. Specify the importer license.
19
18
#. Implement the ``advisory_data`` method to process the data source you're writing an importer for.
20
19
#. Add the newly created importer to the importers registry at
21
20
``vulnerabilites/importers/__init__.py``
@@ -45,24 +44,24 @@ VulnerableCode extensively uses Package URLs to identify a package. See the
45
44
AdvisoryData
46
45
^^^^^^^^^^^^^
47
46
48
-
``AdvisoryData`` is an intermediate data-format,
49
-
it is expected, that your importer converts the raw scraped data into ``AdvisoryData`` objects.
50
-
All the fields in ``AdvisoryData`` dataclass are optional, it is the importer's resposibility to
51
-
ensure that it must contain meaningful information about a vulnerability.
47
+
``AdvisoryData`` is an intermediate dataformat:
48
+
it is expected that your importer will convert the raw scraped data into ``AdvisoryData`` objects.
49
+
All the fields in ``AdvisoryData`` dataclass are optional; it is the importer's resposibility to
50
+
ensure that it contains meaningful information about a vulnerability.
52
51
53
52
AffectedPackage
54
53
^^^^^^^^^^^^^^^^
55
54
56
55
``AffectedPackage`` data type is used to store a range of affected versions and a fixed version of a
57
-
given package. For all versionrelated data, `univers <https://github.com/nexB/univers>`_ library
56
+
given package. For all version-related data, `univers <https://github.com/nexB/univers>`_ library
58
57
is used.
59
58
60
59
Univers
61
60
^^^^^^^^
62
61
63
-
`univers <https://github.com/nexB/univers>`_ is a python implementation of the `vers specification <https://github.com/package-url/purl-spec/pull/139>`_.
64
-
It can parse and compare all the package versions and all the ranges.
65
-
From debian, npm, pypi, ruby and more.
62
+
`univers <https://github.com/nexB/univers>`_ is a Python implementation of the `vers specification <https://github.com/package-url/purl-spec/pull/139>`_.
63
+
It can parse and compare all the package versions and all the ranges,
64
+
from debian, npm, pypi, ruby and more.
66
65
It processes all the version range specs and expressions.
67
66
68
67
Importer
@@ -90,24 +89,24 @@ implementing the unimplemented methods.
90
89
Specify the Importer License
91
90
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
92
91
93
-
Importers scrape data off the internet, in order to make sure the data is useable, a license must be
94
-
provided.
95
-
Populate the ``spdx_license_expression`` with appropriate value.
96
-
The SPDX license identifies can be found at https://spdx.org/licenses/
92
+
Importers scrape data off the internet. In order to make sure the data is useable, a license
93
+
must be provided.
94
+
Populate the ``spdx_license_expression`` with the appropriate value.
95
+
The SPDX license identifiers can be found at https://spdx.org/licenses/.
97
96
98
97
.. note::
99
98
An SPDX license identifier by itself is a valid licence expression. In case you need more complex
expressions, see https://spdx.github.io/spdx-spec/SPDX-license-expressions/
101
100
102
101
Implement the ``advisory_data`` Method
103
102
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
104
103
105
-
The ``advisory_data`` method scrapes the advisories from the data source this importer is targeted
106
-
at.
107
-
It is required to return an *Iterable of AdvisoryData objects*, thus it is a good idea to yield from
108
-
this method after creating each AdvisoryData object
104
+
The ``advisory_data`` method scrapes the advisories from the data source this importer is
105
+
targeted at.
106
+
It is required to return an *Iterable of AdvisoryData objects*, and thus it is a good idea to yield
107
+
from this method after creating each AdvisoryData object.
109
108
110
-
At this point, an example importer will look like:
109
+
At this point, an example importer will look like this:
111
110
112
111
:file:`vulnerabilites/importers/example.py`
113
112
@@ -133,11 +132,11 @@ This importer is only a valid skeleton and does not import anything at all.
133
132
Let us implement another dummy importer that actually imports some data.
134
133
135
134
Here we have a ``dummy_package`` which follows ``NginxVersionRange`` and ``SemverVersion`` for
136
-
version management from `univers <https://github.com/nexB/univers>`_
135
+
version management from `univers <https://github.com/nexB/univers>`_.
137
136
138
137
.. note::
139
138
140
-
It is possible that versioning scheme you are targetting has not yet been implemented in the `univers <https://github.com/nexB/univers>`_ library. If this is the case, you'll need to head over over there and implement one.
139
+
It is possible that the versioning scheme you are targetting has not yet been implemented in the `univers <https://github.com/nexB/univers>`_ library. If this is the case, you'll need to head over there and implement one.
141
140
142
141
.. code-block:: python
143
142
@@ -241,7 +240,7 @@ Congratulations! You've written your first importer.
241
240
Run Your First Importer
242
241
^^^^^^^^^^^^^^^^^^^^^^^^^^
243
242
244
-
If everything went fine, you'll see your importer in the list of available importers
243
+
If everything went well, you'll see your importer in the list of available importers.
245
244
246
245
.. code-block:: console
247
246
:emphasize-lines: 5
@@ -252,7 +251,7 @@ If everything went fine, you'll see your importer in the list of available impor
252
251
vulnerabilities.importers.nginx.NginxImporter
253
252
vulnerabilities.importers.example.ExampleImporter
254
253
255
-
Now, run the importer
254
+
Now, run the importer.
256
255
257
256
.. code-block:: console
258
257
@@ -285,7 +284,7 @@ For more visibility, turn on debug logs in :file:`vulnerablecode/settings.py`.
285
284
},
286
285
}
287
286
288
-
Invoke the import command now and you'll see (in a fresh database)
287
+
Invoke the import command now and you'll see (in a fresh database):
0 commit comments