#1300 | Add Qualifire integration #1311

yuval-qf · 2025-09-01T11:48:07Z

Description

Motivation

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Refactoring (no functional changes)

How Has This Been Tested?

Unit Tests
Integration Tests
Manual Testing

Screenshots (if applicable)

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Related Issues

matter-code-review · 2025-09-01T11:48:54Z

Description

Summary By MatterAI

🔄 What Changed

This pull request introduces Qualifire integration by adding support for Anthropic beta features. Specifically, it updates the package.json version to 1.12.3, adds anthropicBeta header extraction in src/handlers/handlerUtils.ts, refines error handling and introduces a new helper function createContentBlockStartEvent in src/providers/bedrock/messages.ts for standardized content block event creation in streaming responses. Additionally, src/providers/google-vertex-ai/api.ts is updated to conditionally propagate the anthropic-beta header to upstream requests.

🔍 Impact of the Change

These changes enable the gateway to support Anthropic beta features, crucial for Qualifire integration, by correctly parsing and forwarding relevant headers. The Bedrock provider benefits from more robust error handling for non-200 responses and improved modularity in stream chunk processing. The version bump reflects ongoing development.

📁 Total Files Changed

package.json: Version updated from 1.12.2 to 1.12.3.
src/handlers/handlerUtils.ts: Added anthropicBeta header extraction from x-${POWERED_BY}-anthropic-beta.
src/providers/bedrock/messages.ts: Modified error handling logic and introduced createContentBlockStartEvent function for stream chunk processing.
src/providers/google-vertex-ai/api.ts: Enhanced header generation to include anthropic-beta based on provider options or request body.

🧪 Test Added

N/A

🔒Security Vulnerabilities

No new security vulnerabilities were introduced. The improved error handling in Bedrock messages can indirectly enhance reliability.

Motivation

N/A

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Refactoring (no functional changes)

How Has This Been Tested?

Unit Tests
Integration Tests
Manual Testing

Screenshots (if applicable)

N/A

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Related Issues

N/A

Tip

Quality Recommendations

Add unit tests for the createContentBlockStartEvent function in src/providers/bedrock/messages.ts to ensure correct event generation for different content block types (tool use, thinking, redacted thinking).
Consider adding a dedicated type or a helper function for extracting anthropicBeta from providerOptions and gatewayRequestBody in src/providers/google-vertex-ai/api.ts to improve type safety and reduce potential for typos or inconsistent access patterns.
Review the error handling change in src/providers/bedrock/messages.ts from if (responseStatus !== 200 && 'error' in response) to if (responseStatus !== 200) to confirm BedrockErrorResponseTransform and generateInvalidProviderResponseError can robustly handle all possible non-200 responses, especially those that might not strictly conform to BedrockErrorResponse structure.

Tanka Poem ♫

New beta flows bloom,
Headers whisper secrets now,
Bedrock streams refined,
Gateway grows, a steady hum,
Code evolves, a future bright. ✨

Sequence Diagram

sequenceDiagram
    participant Client
    participant Gateway as Portkey Gateway
    participant HandlerUtils as handlerUtils.ts
    participant Bedrock as Bedrock Provider
    participant VertexAI as Google Vertex AI Provider

    Client->>Gateway: POST /v1/chat/completions (x-portkey-anthropic-beta: value)
    Gateway->>HandlerUtils: constructConfigFromRequestHeaders(requestHeaders)
    Note over HandlerUtils: Extracts anthropicBeta from header
    HandlerUtils-->>Gateway: providerOptions { anthropicBeta: value }

    alt Request to Google Vertex AI
        Gateway->>VertexAI: headers({c, providerOptions, gatewayRequestBody})
        Note over VertexAI: Checks providerOptions.anthropicBeta or gatewayRequestBody.anthropic_beta
        VertexAI-->>Gateway: headers { 'anthropic-beta': value }
        Gateway->>VertexAI: POST /v1/models/gemini:generateContent (with anthropic-beta header)
        VertexAI-->>Gateway: Response
    else Request to Bedrock (Streaming)
        Gateway->>Bedrock: POST /messages (streaming)
        Bedrock-->>Gateway: Stream Chunks (BedrockChatCompleteStreamChunk)
        Gateway->>Gateway: BedrockConverseMessagesStreamChunkTransform(responseChunk, fallbackId)
        Gateway->>Gateway: createContentBlockStartEvent(parsedChunk)
        Note over Gateway: Generates RawContentBlockStartEvent for tool_use, thinking, or redacted_thinking
        Gateway-->>Client: Transformed Stream Chunks (event: content_block_start, etc.)
    end

    alt Error Handling in Bedrock
        Bedrock-->>Gateway: Non-200 Error Response
        Gateway->>Gateway: BedrockErrorResponseTransform(response as BedrockErrorResponse)
        Gateway-->>Client: ErrorResponse
    end

matter-code-review

Comprehensive Qualifire integration with 15 guardrail handlers. Found critical bugs in error handling patterns and dead code issues.

Skipped files

conf.json: Skipped file pattern
plugins/qualifire/manifest.json: Skipped file pattern
plugins/qualifire/qualifire.test.ts: File hunk diff too large

plugins/qualifire/dangerousContent.ts

plugins/qualifire/grounding.ts

plugins/qualifire/hallucinations.ts

plugins/qualifire/harassment.ts

plugins/qualifire/hateSpeech.ts

plugins/qualifire/promptInjections.ts

plugins/qualifire/sexualContent.ts

plugins/qualifire/sql.ts

plugins/qualifire/toolUseQuality.ts

plugins/qualifire/wordCount.ts

matter-code-review · 2025-09-02T07:16:29Z

Timeout increase from 10s to 60s for Qualifire API calls - reasonable change for external service reliability.

matter-code-review

Good refactoring to eliminate dead variables and improve error handling consistency across Qualifire plugins. One test timeout adjustment needs verification.

plugins/qualifire/qualifire.test.ts

matter-code-review · 2025-09-03T12:06:59Z

Code quality is good overall with proper TypeScript patterns. Found one potential null reference bug in regex plugin and some architectural improvements needed.

matter-code-review · 2025-09-07T14:46:33Z

Code quality is good overall with proper error handling improvements and provider integration. Found one potential null reference bug in stream processing.

narengogi · 2025-09-08T08:20:15Z

plugins/index.ts

 import { handler as defaultalllowercase } from './default/alllowercase';
 import { handler as defaultendsWith } from './default/endsWith';
 import { handler as defaultmodelWhitelist } from './default/modelWhitelist';
+import { handler as qualifireDangerousContent } from './qualifire/dangerousContent';


[recommendation] would recommend creating an index file in you folder and exporting all of these imports from one single file for cleaner code

matter-code-review · 2025-09-17T14:50:18Z

Added Qualifire integration with new request type validation, image editing support, and provider enhancements. Critical security and performance improvements included.

matter-code-review · 2025-09-28T07:47:03Z

Added imageEdit support across providers and updated version. Minor type fix in requestBody.ts.

matter-code-review · 2025-09-29T11:34:44Z

Added Qualifier integration with Bedrock and Vertex AI enhancements

yuval-qf added 11 commits August 25, 2025 18:28

Add qualifire guardrails

c90b35c

Fix harassment typo

a84a6f9

Fix tsq id

2b1a2dc

Fix tsq schema

39af1b1

Remove log

c61801b

Add tests - wip

c38a28b

Add more tests

f55a35d

Rename toolselection -> tool use

086f85e

Use prod url

908ab75

CR and test fixes

4a37301

Merge branch 'main' into feature/add-qualifire-integration

1ec59d3

yuval-qf marked this pull request as ready for review September 1, 2025 11:48

matter-code-review bot reviewed Sep 1, 2025

View reviewed changes

yuval-qf added 2 commits September 2, 2025 10:15

Remove syntax checks

dce0bab

Update default timeout

89a3dc9

AI CR

b0e1227

matter-code-review bot reviewed Sep 2, 2025

View reviewed changes

plugins/qualifire/qualifire.test.ts Show resolved Hide resolved

yuval-qf mentioned this pull request Sep 2, 2025

#1300 | Feature | Add qualifire guardrails qualifire-dev/portkey-gateway#1

Closed

15 tasks

Merge branch 'main' into feature/add-qualifire-integration

45a56ed

Merge branch 'main' into feature/add-qualifire-integration

776f8f2

narengogi approved these changes Sep 8, 2025

View reviewed changes

Merge branch 'main' into feature/add-qualifire-integration

e84af79

Merge branch 'main' into feature/add-qualifire-integration

a65f9d7

Merge branch 'main' into feature/add-qualifire-integration

d3708ad

#1300 | Add Qualifire integration #1311

Are you sure you want to change the base?

#1300 | Add Qualifire integration #1311

Uh oh!

Conversation

yuval-qf commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation

Type of Change

How Has This Been Tested?

Screenshots (if applicable)

Checklist

Related Issues

Uh oh!

matter-code-review bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Summary By MatterAI

🔄 What Changed

🔍 Impact of the Change

📁 Total Files Changed

🧪 Test Added

🔒Security Vulnerabilities

Motivation

Type of Change

How Has This Been Tested?

Screenshots (if applicable)

Checklist

Related Issues

Quality Recommendations

Tanka Poem ♫

Sequence Diagram

Uh oh!

matter-code-review bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matter-code-review bot commented Sep 2, 2025

Uh oh!

matter-code-review bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

matter-code-review bot commented Sep 3, 2025

Uh oh!

matter-code-review bot commented Sep 7, 2025

Uh oh!

narengogi Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

matter-code-review bot commented Sep 17, 2025

Uh oh!

matter-code-review bot commented Sep 28, 2025

Uh oh!

matter-code-review bot commented Sep 29, 2025

Uh oh!

Uh oh!

yuval-qf commented Sep 1, 2025 •

edited

Loading

matter-code-review bot commented Sep 1, 2025 •

edited

Loading