Skip to content

Commit bd7ba7a

Browse files
vrushankportkeyvrushankportkey
authored
Merge pull request #337 from Portkey-AI/chore/add-guardrails-response-schema
Chore/add-guardrails-response-schema-and-cache-note
2 parents 146f04c + 31c25b5 commit bd7ba7a

File tree

2 files changed

+308
-0
lines changed

2 files changed

+308
-0
lines changed

‎product/ai-gateway/cache-simple-and-semantic.mdx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ Portkey cache serves requests upto **20x times faster** and **cheaper**.
1818

1919
To enable Portkey cache, just add the `cache` params to your [config object](/api-reference/config-object#cache-object-details).
2020

21+
<Note>
22+
Caching will not work if the `x-portkey-debug: "false"` header is included in the request
23+
</Note>
24+
2125
## Simple Cache
2226

2327

‎product/guardrails.mdx

Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,308 @@ Portkey will also show the feedback object logged for each request
330330

331331
---
332332

333+
334+
335+
336+
337+
338+
339+
340+
## Understanding Guardrail Response Structure
341+
342+
When Guardrails are enabled and configured to run synchronously (`async=false`), Portkey adds a `hook_results` object to your API responses. This object provides detailed information about the guardrail checks that were performed and their outcomes.
343+
344+
### Hook Results Structure
345+
346+
The `hook_results` object contains two main sections:
347+
348+
```json
349+
"hook_results": {
350+
"before_request_hooks": [...], // Guardrails applied to the input
351+
"after_request_hooks": [...] // Guardrails applied to the output
352+
}
353+
```
354+
355+
Each section contains an array of guardrail execution results, structured as follows:
356+
357+
<Expandable title="Hook Result Object Structure">
358+
<ResponseField name="verdict" type="boolean">
359+
Overall verdict of this guardrail (true = passed, false = failed). Only true if all checks within this guardrail passed.
360+
</ResponseField>
361+
<ResponseField name="id" type="string">
362+
The ID of the guardrail that was executed (e.g., "pg-check-cfa540")
363+
</ResponseField>
364+
<ResponseField name="transformed" type="boolean">
365+
Indicates if the guardrail modified the request or response (e.g., PII redaction)
366+
</ResponseField>
367+
<ResponseField name="checks" type="array">
368+
An array of individual check results within this guardrail
369+
<Expandable title="Check Object Structure">
370+
<ResponseField name="data" type="object">
371+
Check-specific data that varies based on the guardrail type (e.g., for wordCount it includes counts and thresholds)
372+
</ResponseField>
373+
<ResponseField name="verdict" type="boolean">
374+
Result of this specific check (true = passed, false = failed)
375+
</ResponseField>
376+
<ResponseField name="id" type="string">
377+
Identifier of the specific check (e.g., "default.sentenceCount", "default.wordCount")
378+
</ResponseField>
379+
<ResponseField name="execution_time" type="number">
380+
Time taken to execute this check in milliseconds
381+
</ResponseField>
382+
<ResponseField name="transformed" type="boolean">
383+
Whether this check modified the content
384+
</ResponseField>
385+
<ResponseField name="created_at" type="string">
386+
Timestamp when the check was executed
387+
</ResponseField>
388+
<ResponseField name="log" type="string|null">
389+
Additional logging information (if available)
390+
</ResponseField>
391+
<ResponseField name="error" type="string|object">
392+
Error message or object if the guardrail encountered an error
393+
</ResponseField>
394+
</Expandable>
395+
</ResponseField>
396+
<ResponseField name="feedback" type="object">
397+
Feedback information configured in the guardrail
398+
<Expandable title="Feedback Object Structure">
399+
<ResponseField name="value" type="number">
400+
The numerical feedback value
401+
</ResponseField>
402+
<ResponseField name="weight" type="number">
403+
The weight assigned to this feedback
404+
</ResponseField>
405+
<ResponseField name="metadata" type="object">
406+
Additional metadata including which checks succeeded or failed
407+
<ResponseField name="successfulChecks" type="string">
408+
Comma-separated list of checks that passed
409+
</ResponseField>
410+
<ResponseField name="failedChecks" type="string">
411+
Comma-separated list of checks that failed
412+
</ResponseField>
413+
<ResponseField name="erroredChecks" type="string">
414+
Comma-separated list of checks that encountered errors
415+
</ResponseField>
416+
</ResponseField>
417+
</Expandable>
418+
</ResponseField>
419+
<ResponseField name="execution_time" type="number">
420+
Total execution time for the guardrail in milliseconds
421+
</ResponseField>
422+
<ResponseField name="async" type="boolean">
423+
Whether the guardrail was run asynchronously
424+
</ResponseField>
425+
<ResponseField name="type" type="string">
426+
Always "guardrail" for guardrail hooks
427+
</ResponseField>
428+
<ResponseField name="created_at" type="string">
429+
Timestamp when the guardrail was executed
430+
</ResponseField>
431+
<ResponseField name="deny" type="boolean">
432+
Whether failed checks should deny the request/response
433+
</ResponseField>
434+
</Expandable>
435+
436+
### Example Hook Results
437+
438+
Here's a simplified example of what the `hook_results` might look like in a response:
439+
440+
```json [expandable]
441+
"hook_results": {
442+
"before_request_hooks": [
443+
{
444+
"verdict": true,
445+
"id": "sentence_and_word_check_guardrail",
446+
"transformed": false,
447+
"checks": [
448+
{
449+
"data": {
450+
"sentenceCount": 1,
451+
"minCount": 1,
452+
"maxCount": 99999,
453+
"not": false,
454+
"verdict": true,
455+
"explanation": "The sentence count (1) is within the specified range of 1 to 99999.",
456+
"textExcerpt": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number,..."
457+
},
458+
"verdict": true,
459+
"id": "default.sentenceCount",
460+
"execution_time": 0,
461+
"transformed": false,
462+
"created_at": "2025-05-20T08:00:59.492Z",
463+
"log": null
464+
},
465+
{
466+
"data": {
467+
"wordCount": 24,
468+
"minWords": 1,
469+
"maxWords": 99999,
470+
"not": false,
471+
"verdict": true,
472+
"explanation": "The text contains 24 words, which is within the specified range of 1-99999 words.",
473+
"textExcerpt": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number,..."
474+
},
475+
"verdict": true,
476+
"id": "default.wordCount",
477+
"execution_time": 0,
478+
"transformed": false,
479+
"created_at": "2025-05-20T08:00:59.492Z",
480+
"log": null
481+
}
482+
],
483+
"feedback": {
484+
"value": 5,
485+
"weight": 1,
486+
"metadata": {
487+
"successfulChecks": "default.sentenceCount, default.wordCount",
488+
"failedChecks": "",
489+
"erroredChecks": ""
490+
}
491+
},
492+
"execution_time": 0,
493+
"async": false,
494+
"type": "guardrail",
495+
"created_at": "2025-05-20T08:00:59.492Z",
496+
"deny": false
497+
},
498+
{
499+
"verdict": true,
500+
"id": "character_check_guardrai",
501+
"transformed": false,
502+
"checks": [
503+
{
504+
"data": {
505+
"characterCount": 130,
506+
"minCharacters": 1,
507+
"maxCharacters": 9999999,
508+
"not": false,
509+
"verdict": true,
510+
"explanation": "The text contains 130 characters, which is within the specified range of 1-9999999 characters.",
511+
"textExcerpt": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number,..."
512+
},
513+
"verdict": true,
514+
"id": "default.characterCount",
515+
"execution_time": 0,
516+
"transformed": false,
517+
"created_at": "2025-05-20T08:00:59.492Z",
518+
"log": null
519+
}
520+
],
521+
"feedback": {
522+
"value": 5,
523+
"weight": 1,
524+
"metadata": {
525+
"successfulChecks": "default.characterCount",
526+
"failedChecks": "",
527+
"erroredChecks": ""
528+
}
529+
},
530+
"execution_time": 0,
531+
"async": false,
532+
"type": "guardrail",
533+
"created_at": "2025-05-20T08:00:59.492Z",
534+
"deny": false
535+
}
536+
],
537+
"after_request_hooks": [
538+
{
539+
"verdict": true,
540+
"id": "sentence_and_word_check_guardrail",
541+
"transformed": false,
542+
"checks": [
543+
{
544+
"data": {
545+
"sentenceCount": 2,
546+
"minCount": 1,
547+
"maxCount": 99999,
548+
"not": false,
549+
"verdict": true,
550+
"explanation": "The sentence count (2) is within the specified range of 1 to 99999.",
551+
"textExcerpt": "I'm unable to provide real-time flight information, such as specific flight times, numbers, or bagga..."
552+
},
553+
"verdict": true,
554+
"id": "default.sentenceCount",
555+
"execution_time": 0,
556+
"transformed": false,
557+
"created_at": "2025-05-20T08:01:02.229Z",
558+
"log": null
559+
},
560+
{
561+
"data": {
562+
"wordCount": 46,
563+
"minWords": 1,
564+
"maxWords": 99999,
565+
"not": false,
566+
"verdict": true,
567+
"explanation": "The text contains 46 words, which is within the specified range of 1-99999 words.",
568+
"textExcerpt": "I'm unable to provide real-time flight information, such as specific flight times, numbers, or bagga..."
569+
},
570+
"verdict": true,
571+
"id": "default.wordCount",
572+
"execution_time": 0,
573+
"transformed": false,
574+
"created_at": "2025-05-20T08:01:02.229Z",
575+
"log": null
576+
}
577+
],
578+
"feedback": {
579+
"value": 5,
580+
"weight": 1,
581+
"metadata": {
582+
"successfulChecks": "default.sentenceCount, default.wordCount",
583+
"failedChecks": "",
584+
"erroredChecks": ""
585+
}
586+
},
587+
"execution_time": 0,
588+
"async": false,
589+
"type": "guardrail",
590+
"created_at": "2025-05-20T08:01:02.229Z",
591+
"deny": false
592+
}
593+
]
594+
}
595+
}
596+
```
597+
This example corresponds to a `config` like:
598+
```json
599+
{
600+
"input_guardrails": [
601+
"sentence_and_word_check_guardrail", // Contains sentenceCount and wordCount checks
602+
"characer_check_guardrail" // Contains characterCount check
603+
],
604+
"output_guardrails": [
605+
"sentence_and_word_check_guardrail" // The same guardrail can be used for both input and output
606+
]
607+
}
608+
```
609+
### Important Notes
610+
611+
- If a guardrail is configured to run asynchronously (`async=true`), the `hook_results` will not appear in the API response. The results will only be available in the Portkey logs.
612+
- The `data` field varies based on the type of guardrail check being performed. Each guardrail type returns different information relevant to its function.
613+
- The overall `verdict` for a guardrail is `true` only if all individual checks pass. If any check fails, the verdict will be `false`.
614+
- When `transformed` is `true`, it indicates that the guardrail has modified the content (such as redacting PII).
615+
- If `deny` is `true` and the verdict is `false`, the request will be denied with a 446 status code.
616+
617+
### Special Fields
618+
619+
- **Check-specific data**: Each guardrail type provides different data in the `data` field. For example, a sentence count check provides information about the number of sentences, while a PII check might provide information about detected PII entities.
620+
- **Feedback metadata**: The `metadata` object within `feedback` keeps track of which checks were successful, failed, or encountered errors.
621+
622+
623+
624+
625+
626+
627+
628+
629+
630+
631+
632+
633+
634+
333635
## Defining Guardrails Directly in JSON
334636

335637
On Portkey, you can also create the Guardrails in code and add them to your Configs. Read more about this here:
@@ -338,6 +640,8 @@ On Portkey, you can also create the Guardrails in code and add them to your Conf
338640

339641
---
340642

643+
644+
341645
## Bring Your Own Guardrails
342646

343647
If you already have a custom guardrail pipeline where you send your inputs/outputs for evaluation, you can integrate it with Portkey using a modular, custom webhook! Read more here:

0 commit comments

Comments
 (0)