diff --git a/content/code-security/codeql-cli/codeql-cli-manual/generate-overlay-changes.md b/content/code-security/codeql-cli/codeql-cli-manual/generate-overlay-changes.md new file mode 100644 index 000000000000..6fa8e707e468 --- /dev/null +++ b/content/code-security/codeql-cli/codeql-cli-manual/generate-overlay-changes.md @@ -0,0 +1,101 @@ +--- +title: generate overlay-changes +intro: | + [Plumbing] Generate a file that can be used for the +versions: # DO NOT MANUALLY EDIT. CHANGES WILL BE OVERWRITTEN BY A 🤖 + fpt: '*' + ghec: '*' + ghes: '*' +topics: + - Code Security + - Code scanning + - CodeQL +type: reference +product: '{% data reusables.gated-features.codeql %}' +autogenerated: codeql-cli +--- + + + +{% data reusables.codeql-cli.man-pages-version-note %} + +## Synopsis + +```shell copy +codeql generate overlay-changes [--source-root=] [--output=] ... -- +``` + +## Description + +\[Plumbing] Generate a file that can be used for the +`--overlay-changes` option to +[codeql database create](/code-security/codeql-cli/codeql-cli-manual/database-create) when extracting an overlay database. + +This command is intended to be used mostly for manual or automated +testing. It is not particularly efficient. For production use, consider +if the changes file can instead be derived from something like +`git diff --name-only`. + +## Options + +### Primary Options + +#### `` + +\[Mandatory] Path to the _base_ database into which the overlay will be +extracted. + +#### `-s, --source-root=` + +The directory containing the source code to be extracted as an overlay. +If not given, the current working directory is used. + +#### `-o, --output=` + +The changes file will be written to this location. If it is not +specified, the changes will be written to standard output. + +### Common options + +#### `-h, --help` + +Show this help text. + +#### `-J=` + +\[Advanced] Give option to the JVM running the command. + +(Beware that options containing spaces will not be handled correctly.) + +#### `-v, --verbose` + +Incrementally increase the number of progress messages printed. + +#### `-q, --quiet` + +Incrementally decrease the number of progress messages printed. + +#### `--verbosity=` + +\[Advanced] Explicitly set the verbosity level to one of errors, +warnings, progress, progress+, progress++, progress+++. Overrides `-v` +and `-q`. + +#### `--logdir=` + +\[Advanced] Write detailed logs to one or more files in the given +directory, with generated names that include timestamps and the name of +the running subcommand. + +(To write a log file with a name you have full control over, instead +give `--log-to-stderr` and redirect stderr as desired.) + +#### `--common-caches=` + +\[Advanced] Controls the location of cached data on disk that will +persist between several runs of the CLI, such as downloaded QL packs and +compiled query plans. If not set explicitly, this defaults to a +directory named `.codeql` in the user's home directory; it will be +created if it doesn't already exist. + +Available since `v2.15.2`. diff --git a/content/code-security/codeql-cli/codeql-cli-manual/index.md b/content/code-security/codeql-cli/codeql-cli-manual/index.md index 53c10857ae6d..4eabc073f39f 100644 --- a/content/code-security/codeql-cli/codeql-cli-manual/index.md +++ b/content/code-security/codeql-cli/codeql-cli-manual/index.md @@ -55,6 +55,7 @@ children: - /execute-upgrades - /generate-extensible-predicate-metadata - /generate-log-summary + - /generate-overlay-changes - /generate-query-help - /github-merge-results - /github-upload-results diff --git a/content/code-security/codeql-cli/codeql-cli-manual/resolve-languages.md b/content/code-security/codeql-cli/codeql-cli-manual/resolve-languages.md index e2fdc7c266ca..c363582b0cb4 100644 --- a/content/code-security/codeql-cli/codeql-cli-manual/resolve-languages.md +++ b/content/code-security/codeql-cli/codeql-cli-manual/resolve-languages.md @@ -61,6 +61,12 @@ per-user configuration file). (Note: On Windows the path separator is `;`). +#### `--[no-]filter-to-languages-with-queries` + +List only languages that have default queries. + +Available since `v2.23.0`. + #### `--format=` Select output format. Choices include: diff --git a/content/code-security/codeql-cli/codeql-cli-manual/test-extract.md b/content/code-security/codeql-cli/codeql-cli-manual/test-extract.md index 89cd40f90258..de9a598d5037 100644 --- a/content/code-security/codeql-cli/codeql-cli-manual/test-extract.md +++ b/content/code-security/codeql-cli/codeql-cli-manual/test-extract.md @@ -27,7 +27,7 @@ redirect_from: ## Synopsis ```shell copy -codeql test extract [--source-root=] ... -- +codeql test extract [--print-database] [--source-root=] ... -- ``` ## Description @@ -52,6 +52,11 @@ Override the location of the database being created. By default it will be a subdirectory whose name is derived from the name of the test directory itself with '.testproj' appended. +#### `-D, --print-database` + +Print the path to the database being created, rather than the dataset +inside it. + #### `-s, --source-root=` \[Advanced] The root source code directory, if different from the test diff --git a/content/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-and-the-github-mcp-server.md b/content/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-and-the-github-mcp-server.md index f4ada509f1d7..0f857fe6f5db 100644 --- a/content/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-and-the-github-mcp-server.md +++ b/content/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-and-the-github-mcp-server.md @@ -21,7 +21,7 @@ Push protection prevents you from inadvertently exposing secrets, such as tokens When you're interacting with the {% data variables.product.github %} MCP server, push protection blocks secrets in AI-generated responses as well as preventing secrets from being included in any actions you perform, such as creating an issue. -This protection is on by default for all interactions between the {% data variables.product.github %} MCP server and **public repositories** only, regardless of whether push protection is enabled on the repository's security settings page. +This protection is on by default for all interactions between the {% data variables.product.github %} MCP server and **public repositories**; and between the {% data variables.product.github %} MCP server and private repositories covered by {% data variables.product.prodname_GHAS %}, regardless of whether push protection is enabled on the repository's security settings page. ## Resolving a block diff --git a/content/copilot/concepts/about-mcp.md b/content/copilot/concepts/about-mcp.md index 71aa1dbf3ad9..5f00fff46fe5 100644 --- a/content/copilot/concepts/about-mcp.md +++ b/content/copilot/concepts/about-mcp.md @@ -31,10 +31,11 @@ The {% data variables.product.github %} MCP server is a Model Context Protocol ( * Automate and streamline code-related tasks. * Connect third-party tools (like Cursor, Windsurf, or future integrations) to leverage {% data variables.product.github %}’s context and AI capabilities. * Enable cloud-based workflows that work from any device, without local setup. +* Invoke {% data variables.product.github %} tools, such as {% data variables.copilot.copilot_coding_agent %} (requires {% data variables.product.prodname_copilot %} subscription) and {% data variables.product.prodname_code_scanning %} (requires {% data variables.product.prodname_GHAS %} subscription), to assist with code generation and security analysis. -For public repositories, interactions with the {% data variables.product.github %} MCP server are secured by push protection, which blocks secrets from being included in AI-generated responses and prevents you from exposing secrets through any actions you perform using the server, such as creating an issue. See [AUTOTITLE](/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-and-the-github-mcp-server). +For all public repositories, and private repositories covered by {% data variables.product.prodname_GHAS %}, interactions with the {% data variables.product.github %} MCP server are secured by push protection, which blocks secrets from being included in AI-generated responses and prevents you from exposing secrets through any actions you perform using the server, such as creating an issue. See [AUTOTITLE](/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-and-the-github-mcp-server). -You can access the {% data variables.product.github %} MCP server remotely through {% data variables.product.prodname_vscode %}, or other editors that support remote MCP; or you can run it locally in any MCP-compatible editor, allowing you to choose between the convenience of a hosted solution or the customizability of a self-hosted setup. +You can access the {% data variables.product.github %} MCP server remotely through {% data variables.copilot.copilot_chat_short %} in {% data variables.product.github %}, {% data variables.product.prodname_vscode %}, or other editors that support remote MCP; or you can run it locally in any MCP-compatible editor, allowing you to choose between the convenience of a hosted solution or the customizability of a self-hosted setup. If you want to utilize the remote {% data variables.product.github %} MCP server, you can do so in a few steps, without any local setup. This is particularly useful for users who want to quickly leverage {% data variables.product.github %}’s AI capabilities without the overhead of managing a local MCP server. @@ -46,7 +47,7 @@ To learn how to set up and use the {% data variables.product.github %} MCP serve There is currently broad support for local MCP servers in clients such as {% data variables.product.prodname_vscode %}, JetBrains IDEs, XCode, and others. -Support for remote MCP servers is growing, with editors like {% data variables.product.prodname_vscode %} (with OAuth or PAT), {% data variables.product.prodname_vs %} (PAT only), JetBrains IDEs (PAT only), Xcode (PAT only), Eclipse (PAT only), Windsurf (PAT only), and Cursor (PAT only) already providing this functionality. +Support for remote MCP servers is growing, with editors like {% data variables.product.prodname_vscode %}, {% data variables.product.prodname_vs %}, JetBrains IDEs, Xcode, Eclipse, and Cursor providing this functionality with OAuth or PAT, and Windsurf supporting PAT only. To find out if your preferred editor supports remote MCP servers, check the documentation for your specific editor. diff --git a/content/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-enterprise-policies.md b/content/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-enterprise-policies.md index fe8307d6d418..f0e2ccff2407 100644 --- a/content/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-enterprise-policies.md +++ b/content/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-enterprise-policies.md @@ -32,6 +32,8 @@ Enterprise owners can define a policy for the whole enterprise, or delegate the * Click the **Models** tab to edit the policies that control availability of models beyond the basic models provided with {% data variables.product.prodname_copilot_short %}, which may incur additional costs. 1. For each policy you want to configure, click the dropdown menu and select an enforcement option. Select **No policy** to delegate the decision to individual organization owners. For more information, see [AUTOTITLE](/copilot/reference/feature-availability-enterprise). +{% data reusables.copilot.mcp-servers-policy-note %} + ## Opting in to previews or feedback If your enterprise has a {% data variables.copilot.copilot_business_short %} or {% data variables.copilot.copilot_enterprise_short %} plan and you enable "{% data variables.product.prodname_copilot_short %} in {% data variables.product.prodname_dotcom_the_website %}" on the "Policies" tab, two additional options are displayed: diff --git a/content/copilot/how-tos/provide-context/use-mcp/use-the-github-mcp-server.md b/content/copilot/how-tos/provide-context/use-mcp/use-the-github-mcp-server.md index a48c0aee18ac..d599f2420d94 100644 --- a/content/copilot/how-tos/provide-context/use-mcp/use-the-github-mcp-server.md +++ b/content/copilot/how-tos/provide-context/use-mcp/use-the-github-mcp-server.md @@ -15,11 +15,7 @@ redirect_from: contentType: how-tos --- ->[!NOTE] -> * The remote {% data variables.product.github %} MCP server is currently in {% data variables.release-phases.public_preview %} and subject to change; use of the {% data variables.product.github %} MCP server locally is generally available (GA). -> * MCP support is generally available (GA) in {% data variables.product.prodname_copilot_short %} for {% data variables.product.prodname_vscode %}, {% data variables.product.prodname_vs %}, JetBrains, Eclipse, and Xcode. -> * The **MCP servers in {% data variables.product.prodname_copilot_short %}** policy for enterprises and organizations, disabled by default, controls the use of MCP. -> * While in {% data variables.release-phases.public_preview %}, access to the remote {% data variables.product.github %} MCP server through OAuth in {% data variables.product.prodname_copilot_short %} is governed by the {% data variables.product.prodname_copilot_short %} **Editor preview features** policy at the organization or enterprise level. PAT access to the server is managed by PAT policies. +The {% data variables.product.github %} MCP server is available to all {% data variables.product.github %} users regardless of plan type. However, specific tools within the MCP server inherit the same access requirements as their corresponding {% data variables.product.github %} features. If a feature requires a paid {% data variables.product.github %} or {% data variables.product.prodname_copilot_short %} license, the equivalent MCP tool will require the same subscription. For example, tools that interact with {% data variables.product.prodname_copilot_short %} Coding Agent require a paid {% data variables.product.prodname_copilot_short %} license. {% vscode %} @@ -41,8 +37,7 @@ You can choose to set up the {% data variables.product.github %} MCP server eith The remote {% data variables.product.github %} MCP server uses one-click OAuth authentication by default, but you can also manually configure it to use a {% data variables.product.pat_generic %} (PAT) for authentication. If you use OAuth, the MCP server can only access the scopes you approve during sign-in. In organization-owned contexts, access may also be limited by admin policies that control which scopes and apps are permitted. If you use a PAT, the MCP server will have access to the scopes granted by the PAT, which is also subject to any PAT restrictions configured by the organization. -> [!NOTE] -> If you are an {% data variables.product.prodname_emu %}, then PAT is disabled by default, unless enabled by an enterprise administrator. If PAT is disabled, you won't be able to use PAT authentication. If you have OAuth access policy restrictions, you will need the OAuth App for each client (MCP host application) to be enabled (except {% data variables.product.prodname_vscode %} and {% data variables.product.prodname_vs %}). +If you are an {% data variables.product.prodname_emu %}, then PAT is disabled by default, unless enabled by an enterprise administrator. If PAT is disabled, you won't be able to use PAT authentication. If you have OAuth access policy restrictions, you will need the OAuth App for each client (MCP host application) to be enabled (except {% data variables.product.prodname_vscode %} and {% data variables.product.prodname_vs %} since they are first-party Microsoft IDEs with native {% data variables.product.prodname_copilot_short %} integration). * [Remote MCP server configuration with OAuth](#remote-mcp-server-configuration-with-oauth) * [Remote MCP server configuration with PAT](#remote-mcp-server-configuration-with-pat) @@ -467,6 +462,7 @@ The {% data variables.product.github %} MCP server enables you to perform a wide {% data reusables.copilot.eclipse-prerequisites %} * **Latest version of the {% data variables.product.prodname_copilot %} extension**. Download this from the [Eclipse Marketplace](https://aka.ms/copiloteclipse). For more information, see [AUTOTITLE](/copilot/managing-copilot/configure-personal-settings/installing-the-github-copilot-extension-in-your-environment?tool=eclipse). * **Sign in to {% data variables.product.company_short %} from Eclipse**. +* {% data reusables.copilot.mcp-policy-requirement %} ## Setting up the {% data variables.product.github %} MCP server in Eclipse @@ -534,8 +530,6 @@ The {% data variables.product.github %} MCP server enables you to perform a wide {% webui %} ->[!NOTE] MCP in {% data variables.copilot.copilot_chat_dotcom_short %} is currently in {% data variables.release-phases.public_preview %} and subject to change. - ## About MCP in {% data variables.copilot.copilot_chat_dotcom_short %} The {% data variables.product.github %} MCP server is a Model Context Protocol (MCP) server provided and maintained by {% data variables.product.github %}. MCP allows you to integrate AI capabilities with other tools and services, enhancing your development experience by providing context-aware AI assistance. @@ -553,7 +547,6 @@ The {% data variables.product.github %} MCP server is automatically configured i Some examples of requests you can make are: * `Create a new branch called [BRANCH-NAME] in the repository [USERNAME/REPO-NAME].` - * `Create a new branch called [BRANCH-NAME] in the repository [USERNAME/REPO-NAME].` * `Merge the pull request [PULL-REQUEST-NUMBER] in the repository [USERNAME/REPO-NAME].` 1. {% data variables.copilot.copilot_chat_short %} will ask you to confirm that you want to proceed with the action. Click **Allow** to confirm. diff --git a/content/github-models/use-github-models/evaluating-ai-models.md b/content/github-models/use-github-models/evaluating-ai-models.md index 8347ce40c622..4f7cc4580daf 100644 --- a/content/github-models/use-github-models/evaluating-ai-models.md +++ b/content/github-models/use-github-models/evaluating-ai-models.md @@ -131,11 +131,13 @@ After applying the parameters, you can add additional columns to compare more mo Once the prompt is configured, run a structured evaluation to compare model outputs using real data and repeatable metrics. -Model evaluation helps you understand how different models and prompt configurations perform across real inputs. In the Prompt view, you can apply evaluators to multiple models side by side and review metrics such as similarity, relevance, and groundedness. +Model evaluation helps you understand how different models and prompt configurations perform across real inputs. In the Prompt view, you can apply evaluators to multiple models side by side and review metrics such as similarity, fluency, coherence, relevance, and groundedness. The following evaluators are available: * **Similarity**: Measures how closely a model's output matches an expected or reference answer. This is useful when you want to confirm that the model returns consistent and accurate responses aligned with a known result. The score ranges from 0 to 1, with higher values indicating greater similarity. +* **Fluency**: Evaluates the linguistic quality of a response, including grammar, coherence, and readability. This results in linguistically correct responses. +* **Coherence**: Assesses the ability of the LLM to generate text that reads naturally, flows smoothly, and resembles human-like language in its responses. Use it when assessing the readability and user-friendliness of a model’s generated responses in real-world applications. * **Relevance**: Refers to how effectively a response addresses a question. It assesses the accuracy, completeness, and direct relevance of the response based solely on the given information. The score ranges from 0 to 1, with higher values indicating stronger alignment with the input's intent. * **Groundedness**: Measures how well an answer is anchored in the provided context, evaluating its relevance, accuracy, and completeness based exclusively on that context. It assesses the extent to which the answer fully addresses the question without introducing unrelated or incorrect information. The score ranges from 0 to 1, with higher values indicating higher accuracy. * **Custom prompt**: Lets you define your own evaluation criteria for one LLM to assess the output of another. This allows you to score model outputs based on your own guidelines. You can choose between pass/fail or scored evaluations, making it ideal for scenarios where standard metrics do not capture testing expectations. diff --git a/data/reusables/code-scanning/codeql-query-tables/java.md b/data/reusables/code-scanning/codeql-query-tables/java.md index 27683694bfa8..1f4f6aef33f5 100644 --- a/data/reusables/code-scanning/codeql-query-tables/java.md +++ b/data/reusables/code-scanning/codeql-query-tables/java.md @@ -18,6 +18,7 @@ | [Disabled Netty HTTP header validation](https://codeql.github.com/codeql-query-help/java/java-netty-http-request-or-response-splitting/) | 093, 113 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | | [Disabled Spring CSRF protection](https://codeql.github.com/codeql-query-help/java/java-spring-disabled-csrf-protection/) | 352 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | | [Exposed Spring Boot actuators](https://codeql.github.com/codeql-query-help/java/java-spring-boot-exposed-actuators/) | 200 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "x" aria-label="Not included" %} | +| [Exposed Spring Boot actuators in configuration file](https://codeql.github.com/codeql-query-help/java/java-spring-boot-exposed-actuators-config/) | 200 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "x" aria-label="Not included" %} | | [Expression language injection (JEXL)](https://codeql.github.com/codeql-query-help/java/java-jexl-expression-injection/) | 094 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | | [Expression language injection (MVEL)](https://codeql.github.com/codeql-query-help/java/java-mvel-expression-injection/) | 094 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | | [Expression language injection (Spring)](https://codeql.github.com/codeql-query-help/java/java-spel-expression-injection/) | 094 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | diff --git a/data/reusables/code-scanning/codeql-query-tables/rust.md b/data/reusables/code-scanning/codeql-query-tables/rust.md index aef0a53259c3..08626df0693c 100644 --- a/data/reusables/code-scanning/codeql-query-tables/rust.md +++ b/data/reusables/code-scanning/codeql-query-tables/rust.md @@ -14,5 +14,6 @@ | [Use of a broken or weak cryptographic algorithm](https://codeql.github.com/codeql-query-help/rust/rust-weak-cryptographic-algorithm/) | 327 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "x" aria-label="Not included" %} | | [Use of a broken or weak cryptographic hashing algorithm on sensitive data](https://codeql.github.com/codeql-query-help/rust/rust-weak-sensitive-data-hashing/) | 327, 328, 916 | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "x" aria-label="Not included" %} | | [Access of a pointer after its lifetime has ended](https://codeql.github.com/codeql-query-help/rust/rust-access-after-lifetime-ended/) | 825 | {% octicon "x" aria-label="Not included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "x" aria-label="Not included" %} | +| [Log injection](https://codeql.github.com/codeql-query-help/rust/rust-log-injection/) | 117 | {% octicon "x" aria-label="Not included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "x" aria-label="Not included" %} | {% endrowheaders %} diff --git a/data/reusables/copilot/mcp-servers-policy-note.md b/data/reusables/copilot/mcp-servers-policy-note.md index 4312fb84bf0f..6f4682213fb5 100644 --- a/data/reusables/copilot/mcp-servers-policy-note.md +++ b/data/reusables/copilot/mcp-servers-policy-note.md @@ -1,2 +1,2 @@ > [!NOTE] -> The **MCP servers in {% data variables.product.prodname_copilot_short %}** policy controls use where MCP server support is generally available (GA). In features where MCP support is in preview, for example {% data variables.product.prodname_copilot_short %} editors, availability is controlled by the **Editor preview features** policy. +> The **MCP servers in {% data variables.product.prodname_copilot_short %}** policy controls use where MCP server support is generally available (GA). This policy does not control access and permissions for the {% data variables.product.github %} MCP server in third-party host applications (like Cursor, Windsurf or Claude). For more information on controlling access to the {% data variables.product.github %} MCP server, see the [Policies and Governance](https://github.com/github/github-mcp-server/blob/main/docs/policies-and-governance.md#control-mechanisms) documentation in the {% data variables.product.github %} MCP Server repository.