Using AI for parameter validation

[samshdn]

In Linux, when using the CLI to perform a task, if an incorrect parameter is provided, the system typically responds with a message like: 'unexpected argument "wrong-para" found; tip: a similar argument exists "--parameter".'

In the current project, we need to validate the parameters provided by the user. While we can determine whether a parameter is correct, we do not know the user's intention behind an incorrect parameter, so we cannot provide more helpful information to the user.

[samshdn]

Approach 1:

For this issue, the response provided by the AI is to use the Levenshtein distance algorithm. This algorithm is robust and widely used due to its simplicity and effectiveness in quantifying string differences.

const validParameters = new Set(["refresh", "page"]);

function findClosestMatch(input: string, validOptions: Set<string>): string | null {
    let closestMatch: string | null = null;
    let minDistance = Infinity;

    for (const option of validOptions) {
        const distance = levenshteinDistance(input, option);
        console.log(`Distance between "${input}" and "${option}": ${distance}`);
        if (distance < minDistance) {
            minDistance = distance;
            closestMatch = option;
        }
    }

    return minDistance <= 2 ? closestMatch : null;
}

function levenshteinDistance(a: string, b: string): number {
    const matrix: number[][] = [];

    for (let i = 0; i <= b.length; i++) {
        matrix[i] = [i];
    }

    for (let j = 0; j <= a.length; j++) {
        matrix[0][j] = j;
    }

    for (let i = 1; i <= b.length; i++) {
        for (let j = 1; j <= a.length; j++) {
            if (b.charAt(i - 1) === a.charAt(j - 1)) {
                matrix[i][j] = matrix[i - 1][j - 1];
            } else {
                matrix[i][j] = Math.min(
                    matrix[i - 1][j - 1] + 1,
                    matrix[i][j - 1] + 1,
                    matrix[i - 1][j] + 1
                );
            }
        }
    }

    return matrix[b.length][a.length];
}

const providedParams = new URLSearchParams({ fresh: "true", pag:"1" });
const suggestions = [];

for (const [key] of providedParams) {
    if (!validParameters.has(key)) {
        const suggestion = findClosestMatch(key, validParameters);
        if (suggestion) {
            suggestions.push({ provided: key, suggested: suggestion });
        }
    }
}

console.log(suggestions)

The output of the algorithm is:

[
    {
        "provided": "fresh",
        "suggested": "refresh"
    },
    {
        "provided": "pag",
        "suggested": "page"
    }
]

Approach 2:

Another approach is to use an LLM. I wrote code to interact with Claude 3.5. The following is my prompt message:

    const validParamList = "refresh, page";
    const userInputParamList = "fresh, pag";

    const prompt = `
    Given the following list of valid URL parameters: ${validParamList}
    And the following provided parameters: ${userInputParamList}
    
    Please identify any incorrect parameters and suggest the correct ones.
    Format your response as a JSON array of objects, each with 'provided' and 'suggested' fields.
    If a parameter is correct or you can't suggest a replacement, don't include it in the output.
  `;

The output from the LLM is:

[
    {
        "provided": "fresh",
        "suggested": "refresh"
    },
    {
        "provided": "pag",
        "suggested": "page"
    }
]

The Claude model also uses the Levenshtein distance algorithm to identify incorrect parameters.

[samshdn]

Based on the above information, we can make the following initial conclusion:

1. Both approach can be used to check the incorrect parameters
2. The Levenshtein distance algorithm is a commonly used algorithm in spell-checking applications.
3. Approach 1 is more effective than Approach 2, because the results returned from an LLM usually take longer, depending on the network and the condition of the LLM provider.