You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CodeQL packs are used to create, share, depend on, and run CodeQL queries and libraries. You can publish your own CodeQL packs and download packs created by others. CodeQL packs contain queries, library files, query suites, and metadata.
9
+
10
+
There are two types of CodeQL packs: query packs and library packs.
11
+
12
+
* Query packs are designed to be run. When a query pack is published, the bundle includes all the transitive dependencies and a compilation cache. This ensures consistent and efficient execution of the queries in the pack.
13
+
* Library packs are designed to be used by query packs (or other library packs) and do not contain queries themselves. The libraries are not compiled and there is no compilation cache included when the pack is published.
14
+
15
+
You can use the package management commands in the CodeQL CLI to create CodeQL packs, add dependencies to packs, and install or update dependencies. For more information, see ":ref:`Creating and working with CodeQL packs <creating-and-working-with-codeql-packs>`." You can also publish and download CodeQL packs using the CodeQL CLI. For more information, see ":doc:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`."
16
+
17
+
CodeQL pack structure
18
+
---------------------
19
+
20
+
A CodeQL pack must contain a file called ``qlpack.yml`` in its root directory. In the ``qlpack.yml`` file, the ``name:`` field must have a value that follows the format of ``<scope>/<pack>``, where ``<scope>`` is the GitHub organization or user account that the pack will be published to and ``<pack>`` is the name of the pack. The other
21
+
files and directories within the pack should be logically organized. For example, typically:
22
+
23
+
- Queries are organized into directories for specific categories.
24
+
- Queries for specific products, libraries, and frameworks are organized into
25
+
their own top-level directories.
26
+
27
+
About ``qlpack.yml`` files
28
+
--------------------------
29
+
30
+
When executing query-related commands, CodeQL first looks in siblings of the installation directory (and their subdirectories) for ``qlpack.yml`` files.
31
+
Then it checks the package cache for CodeQL packs which have been downloaded. This means that when you are developing queries locally, the local packages
32
+
in the installation directory override packages of the same name in the package cache, so that you can test your local changes.
33
+
34
+
The metadata in each `qlpack.yml`` file tells
35
+
CodeQL how to compile any queries in the pack, what libraries the pack depends on, and where to
36
+
find query suite definitions.
37
+
38
+
The contents of the CodeQL pack (queries or libraries used in CodeQL analysis) is
39
+
included in the same directory as ``qlpack.yml``, or its subdirectories.
40
+
41
+
The location of ``qlpack.yml`` defines the library path for the content
42
+
of the CodeQL pack. That is, for all ``.ql`` and ``.qll`` files in the pack,
43
+
CodeQL will resolve all import statements relative to the ``qlpack.yml`` at the
44
+
pack's root.
45
+
46
+
.. _codeqlpack-yml-properties:
47
+
48
+
``qlpack.yml`` properties
49
+
~~~~~~~~~~~~~~~~~~~~~~~~~
50
+
51
+
The following properties are supported in ``qlpack.yml`` files.
52
+
53
+
.. list-table::
54
+
:header-rows: 1
55
+
:widths: auto
56
+
57
+
* - Property
58
+
- Example
59
+
- Required
60
+
- Purpose
61
+
* - ``name``
62
+
- ``octo-org/security-queries``
63
+
- All packs
64
+
- The scope, where the CodeQL pack is published, and the name of the pack defined using alphanumeric characters and hyphens. It must be unique as CodeQL cannot differentiate between CodeQL packs with identical names. Name components cannot start or end with a hyphen. Additionally, a period is not allowed in pack names at all. Use the pack name to specify queries to run using ``database analyze`` and to define dependencies between QL packs (see examples below).
65
+
* - ``version``
66
+
- ``0.0.0``
67
+
- All packs
68
+
- A version range for this CodeQL pack. This must be a valid semantic version that meets the `SemVer v2.0.0 specification <https://semver.org/spec/v2.0.0.html>`__.
69
+
* - ``dependencies``
70
+
- ``codeql/javascript-all: ^1.2.3``
71
+
- Optional
72
+
- The names and version ranges of any CodeQL packs that this pack depends on, as a mapping. This gives the pack access to any libraries, database schema, and query suites defined in the dependency. For more information, see `SemVer ranges <https://docs.npmjs.com/cli/v6/using-npm/semver#ranges>`__ in the NPM documentation.
73
+
* - ``suites``
74
+
- ``octo-org-query-suites``
75
+
- Optional
76
+
- The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. QL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. This is not supported for CodeQL packs downloaded from a package registry. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
77
+
* - ``extractor``
78
+
- ``javascript``
79
+
- All test packs
80
+
- The CodeQL language extractor to use when the CLI creates a database in the pack. For more information about testing queries, see ":doc:`Testing custom queries <testing-custom-queries>`."
81
+
* - ``tests``
82
+
- ``.``
83
+
- Optional for test packs
84
+
- The path to a directory within the pack that contains tests, defined relative to the pack directory. Use ``.`` to specify the whole pack. Any queries in this directory are run as tests when ``test run`` is run with the ``--strict-test-discovery`` option. These queries are ignored by query suite definitions that use ``queries`` or ``qlpack`` instructions to ask for all queries in a particular pack.
85
+
* - ``dbscheme``
86
+
- ``semmlecode.python.dbscheme``
87
+
- Core language packs only
88
+
- The path to the :ref:`database schema <codeql-database-schema>` for all libraries and queries written for this CodeQL language (see example below).
89
+
* - ``upgrades``
90
+
- ``.``
91
+
- Core language packs only
92
+
- The path to a directory within the pack that contains upgrade scripts, defined relative to the pack directory. The ``database upgrade`` action uses these scripts to update databases that were created by an older version of an extractor so they're compatible with the current extractor (see `Upgrade scripts for a language <#upgrade-scripts-for-a-language>`__ below.)
- Metadata that will be displayed on the packaging search page in the packages section of the account that the CodeQL pack is published to.
97
+
* - ``licenses``
98
+
- ``(LGPL-2.1 AND MIT)``
99
+
- All packs
100
+
- Metadata that will be displayed on the packaging search page in the packages section of the account that the CodeQL pack is published to. For a list of allowed licenses, see `SPDX License List <https://spdx.org/licenses/>`__ in the SPDX Specification.
101
+
* - ``description``
102
+
- ``Human-readable description of the contents of the CodeQL pack.``
103
+
- All packs
104
+
- Metadata that will be displayed on the packaging search page in the packages section of the account that the CodeQL pack is published to.
Copy file name to clipboardExpand all lines: docs/codeql/codeql-cli/about-ql-packs.rst
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -87,7 +87,7 @@ The following properties are supported in ``qlpack.yml`` files.
87
87
* - ``suites``
88
88
- ``suites``
89
89
- Optional
90
-
- The path to a directory that contains the "well-known" query suites in the pack, defined relative to the pack directory. You can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. To use query suites stored in other directories in the pack, you must provide their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
90
+
- The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. QL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
The ``analyze`` command above runs the default suite from ``microsoft/coding-standards v1.0.0`` and the latest version of ``scope/other-pack`` on the specified database.
125
+
For further information about default suites, see ":ref:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`".
113
126
114
-
To run the GitHub code scanning suite of queries over a CodeQL database for a C/C++ codebase,
127
+
Running query suites
128
+
~~~~~~~~~~~~~~~~~~~~
129
+
130
+
To run a query suite over a CodeQL database for a C/C++ codebase,
115
131
you could use the following command from the directory containing your database::
The analysis generates a file in the v2.1.0 SARIF format that is supported by all versions of GitHub.
120
-
This file can be uploaded to GitHub using ``github upload-results`` or the code scanning API.
121
-
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/secure-coding/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__
136
+
This file can be uploaded to GitHub by executing ``codeql github upload-results`` or the code scanning API.
137
+
For more information, see `Analyzing a CodeQL database <https://docs.github.com/en/code-security/secure-coding/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database>`__
122
138
or `Code scanning API <https://docs.github.com/en/rest/reference/code-scanning>`__ in the GitHub documentation.
123
139
124
-
CodeQL query suites are ``.qls`` files that use directives to select queries to run
140
+
CodeQL query suites are ``.qls`` files that use directives to select queries to run
125
141
based on certain metadata properties. The standard QL packs have metadata that specify
126
-
the location of the code scanning suites, so the CodeQL CLI knows where to find these
142
+
the location of the query suites used by code scanning, so the CodeQL CLI knows where to find these
127
143
suite files automatically, and you don't have to specify the full path on the command line.
128
144
For more information, see ":ref:`About QL packs <standard-ql-packs>`."
129
145
@@ -137,7 +153,7 @@ and at the following path in the CodeQL for Go repository::
137
153
ql/src/codeql-suites/go-code-scanning.qls
138
154
139
155
The repository also includes the query suites used by `LGTM.com <https://lgtm.com>`__.
140
-
These are stored alongside the code scanning suites with names of the form: ``<language>-lgtm.qls``.
156
+
These are stored alongside the query suites for code scanning with names of the form: ``<language>-lgtm.qls``.
141
157
142
158
For information about creating custom query suites, see ":doc:`Creating
@@ -149,12 +165,21 @@ When you create a CodeQL database, the extractor stores diagnostic data in the d
149
165
150
166
If the analysis found fewer results for standard queries than you expected, review the results of the diagnostic and summary queries to check whether the CodeQL database is likely to be a good representation of the codebase that you want to analyze.
151
167
168
+
Integrating a CodeQL pack into a code scanning workflow in GitHub
You can use CodeQL query packs in your Code Scanning setup. This allows you to select query packs published by various sources and use them to analyze your code.
174
+
For more information, see "`Using CodeQL query packs in the CodeQL action <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-codeql-query-packs/>`_" or "`Downloading and using CodeQL query packs in your CI system <https://docs.github.com/en/code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#downloading-and-using-codeql-query-packs>`_."
175
+
176
+
152
177
Running all queries in a directory
153
178
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
154
179
155
180
You can run all the queries located in a directory by providing the directory
156
181
path, rather than listing all the individual query files. Paths are searched
157
-
recursively, so any queries contained in subfolders will also be executed.
182
+
recursively, so any queries contained in subfolders will also be executed.
158
183
159
184
.. pull-quote::
160
185
@@ -164,12 +189,12 @@ recursively, so any queries contained in subfolders will also be executed.
164
189
<about-ql-packs>` when executing ``database analyze``
165
190
as it contains some special queries that aren't designed to be used with
166
191
the command. Rather, to run a wide range of useful queries, run one of the
167
-
LGTM.com query suites.
168
-
192
+
LGTM.com query suites.
193
+
169
194
For example, to execute all Python queries contained in the ``Functions``
Copy file name to clipboardExpand all lines: docs/codeql/codeql-cli/codeql-cli-reference.rst
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,12 +9,14 @@ Learn more about the files you can use when running CodeQL processes and the res
9
9
:titlesonly:
10
10
:hidden:
11
11
12
+
about-codeql-packs
12
13
about-ql-packs
13
14
query-reference-files
14
15
sarif-output
15
16
exit-codes
16
17
17
18
19
+
- :doc:`About CodeQL packs <about-codeql-packs>`: CodeQL packs are created with the CodeQL CLI and are used to create, depend on, publish, and run CodeQL queries and libraries.
18
20
- :doc:`About QL packs <about-ql-packs>`: QL packs are used to organize the files used in CodeQL analysis. They
19
21
contain queries, library files, query suites, and important metadata.
20
22
- :doc:`Query reference files <query-reference-files>`: A query reference file is text file that defines the location of one query to test.
0 commit comments