Skip to content

Mitigate KQL injection in TopicToTableMapping configuration values#155

Draft
Copilot wants to merge 2 commits intomasterfrom
copilot/fix-kql-command-validation
Draft

Mitigate KQL injection in TopicToTableMapping configuration values#155
Copilot wants to merge 2 commits intomasterfrom
copilot/fix-kql-command-validation

Conversation

Copy link
Copy Markdown

Copilot AI commented Apr 4, 2026

Pull Request Description

TopicToTableMapping.validate() only checked for null/empty on db, table, mapping, and format fields. These values are interpolated directly into KQL commands via String.formatted() in KustoSinkTask, enabling injection of arbitrary management commands through KQL metacharacters (;, |, ').

// Before: attacker sets table = "TestTable; .show databases | project DatabaseName"
// This gets interpolated into:
FETCH_TABLE_COMMAND.formatted(table)  // → "TestTable; .show databases | project DatabaseName | count"

Changes:

  • Add allowlist regex validation in TopicToTableMapping.validate() for all fields interpolated into KQL:
    • db, table, mapping: ^[a-zA-Z0-9_.\-]+$
    • format: ^[a-zA-Z0-9]+$
  • Add 8 test cases covering semicolon, pipe, and single-quote injection vectors plus valid identifier acceptance

Future Release Comment

Breaking Changes:

  • db, table, and mapping values in kusto.tables.topics.mapping now reject names containing characters outside [a-zA-Z0-9_.-]. format is restricted to [a-zA-Z0-9]. Configurations using spaces or other special characters in entity names will fail validation at startup.

Features:

  • None

Fixes:

  • Input validation on kusto.tables.topics.mapping fields to prevent KQL command injection (CWE-943)

Copilot AI and others added 2 commits April 4, 2026 04:07
Validate db, table, mapping, and format fields against strict allowlist
patterns to prevent KQL metacharacter injection (semicolons, pipes,
single quotes, etc.) in management commands constructed via
String.formatted() in KustoSinkTask.

- db, table, mapping: Only allow alphanumeric, underscores, hyphens,
  spaces, and dots
- format: Only allow alphanumeric characters
- Add 7 new tests covering injection patterns and valid names

Agent-Logs-Url: https://github.com/Azure/kafka-sink-azure-kusto/sessions/2be3571a-a8d8-49fc-a8d4-c8678f5b1872

Co-authored-by: tanmaya-panda1 <108695755+tanmaya-panda1@users.noreply.github.com>
…e name test

Agent-Logs-Url: https://github.com/Azure/kafka-sink-azure-kusto/sessions/2be3571a-a8d8-49fc-a8d4-c8678f5b1872

Co-authored-by: tanmaya-panda1 <108695755+tanmaya-panda1@users.noreply.github.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Strengthens configuration validation to prevent KQL management-command injection by allowlisting safe characters in TopicToTableMapping fields that are interpolated into KQL, and adds unit tests to cover common metacharacter injection vectors.

Changes:

  • Add allowlist regex validation for db, table, mapping, and format in TopicToTableMapping.validate().
  • Add new unit tests asserting invalid characters (e.g., ;, |, ') are rejected and valid identifiers are accepted.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/main/java/com/microsoft/azure/kusto/kafka/connect/sink/TopicToTableMapping.java Introduces regex allowlist validation for Kusto identifiers and ingestion format to mitigate KQL injection.
src/test/java/com/microsoft/azure/kusto/kafka/connect/sink/KustoSinkConnectorConfigTest.java Adds tests for rejection of injection-like inputs and acceptance of valid identifiers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +99 to +108
validateKustoIdentifier("db", db);
validateKustoIdentifier("table", table);
if (mapping != null && !mapping.isEmpty()) {
validateKustoIdentifier("mapping", mapping);
}
if (format != null && !format.isEmpty()) {
if (!VALID_FORMAT_PATTERN.matcher(format).matches()) {
throw new ConfigException(
String.format("'format' contains invalid characters: '%s'. Only alphanumeric characters are allowed.", format));
}
Copy link

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate() currently only validates format if it is non-null/non-empty. If the config omits format, this passes validation but later code paths call format.equalsIgnoreCase(...) (e.g., in table-access validation), which can throw a NullPointerException. Consider making format required here (throw ConfigException when missing/blank) or ensuring downstream callers handle null safely; add a test for missing/blank format to lock this in.

Suggested change
validateKustoIdentifier("db", db);
validateKustoIdentifier("table", table);
if (mapping != null && !mapping.isEmpty()) {
validateKustoIdentifier("mapping", mapping);
}
if (format != null && !format.isEmpty()) {
if (!VALID_FORMAT_PATTERN.matcher(format).matches()) {
throw new ConfigException(
String.format("'format' contains invalid characters: '%s'. Only alphanumeric characters are allowed.", format));
}
if (null == format || format.isEmpty()) {
throw new ConfigException("'format' must be provided for each mapping");
}
validateKustoIdentifier("db", db);
validateKustoIdentifier("table", table);
if (mapping != null && !mapping.isEmpty()) {
validateKustoIdentifier("mapping", mapping);
}
if (!VALID_FORMAT_PATTERN.matcher(format).matches()) {
throw new ConfigException(
String.format("'format' contains invalid characters: '%s'. Only alphanumeric characters are allowed.", format));

Copilot uses AI. Check for mistakes.
Comment on lines +112 to +116
private static void validateKustoIdentifier(String fieldName, String value) {
if (!VALID_KUSTO_IDENTIFIER_PATTERN.matcher(value).matches()) {
throw new ConfigException(
String.format("'%s' contains invalid characters: '%s'. Only alphanumeric characters, underscores, hyphens, and dots are allowed.",
fieldName, value));
Copy link

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validateKustoIdentifier(...) assumes value is non-null and will throw a NullPointerException if called with null. Since this helper is new and likely to be reused, it would be more robust to explicitly check for null/blank and throw a ConfigException with a clear message (or fold the required/optional logic into the helper).

Copilot uses AI. Check for mistakes.
Comment on lines +191 to +193
@Test
public void shouldRejectTableNameWithSemicolon() {
HashMap<String, String> settings = setupConfigs();
Copy link

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description mentions adding 8 test cases for injection vectors, but this diff adds 7 new tests (5 rejection + 2 acceptance). Either add the missing test case or update the PR description/release notes to match what’s actually covered.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants