Merge branch 'main' into chore/openai-compliance-improvements

Author: narengogi

cookbook/integrations/Sutra_with_Portkey.ipynb

@@ -14,14 +14,18 @@ Check out the [API docs](https://portkey.ai/docs/welcome/make-your-first-request
 
 ## Local Deployment
 
-1. Run through [NPX](#node) or [BunX](#bun) Install
+1. Do [NPM](#node) or [Bun](#bun) Install
 2. Run a [Node.js Server](#nodejs-server)
-3. Deploy using [Docker](#docker)
-4. Deploy using [Docker Compose](#docker-compose)
-5. Deploy on [Cloudflare Workers](#cloudflare-workers)
-6. Deploy on [App Stack](#deploy-to-app-stack)
+3. Deploy on [App Stack](#deploy-to-app-stack)
+4. Deploy on [Cloudflare Workers](#cloudflare-workers)
+5. Deploy using [Docker](#docker)
+6. Deploy using [Docker Compose](#docker-compose)
 7. Deploy on [Replit](#replit)
 8. Deploy on [Zeabur](#zeabur)
+9. Deploy with [Supabase Functions](#supabase-functions)
+10. Deploy using [Fastly](#fastly)
+
+---
 
 ### Node
 
@@ -39,7 +43,37 @@ $ bunx @portkey-ai/gateway
 
 <br>
 
-### Deploy to App Stack  
+
+--- 
+
+### NodeJS Server
+
+1. Clone the Repository
+
+```sh
+git clone https://github.com/portkey-ai/gateway
+```
+
+2. Install the NPM Dependencies
+
+```sh
+cd gateway
+npm i
+npm run build
+```
+
+3. Run the Server
+
+```sh
+node build/start-server.js
+```
+
+<br>
+
+---
+
+### Deploy to App Stack
+
 F5 Distributed Cloud
 1. [Create an App Stack Site](https://docs.cloud.f5.com/docs/how-to/site-management/create-voltstack-site)
 
@@ -94,7 +128,8 @@ curl --request POST \
   --data '{"metadata": {"name": "$DISTRIBUTED_CLOUD_SERVICE_NAME","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","labels": {},"annotations": {},"description": "","disable": false},"spec": {"origin_servers": [{"k8s_service": {"service_name": "$DISTRIBUTED_CLOUD_SERVICE_NAME.$DISTRIBUTED_CLOUD_APP_STACK_NAMESPACE","site_locator": {"site": {"tenant": "$DISTRIBUTED_CLOUD_TENANT_ID","namespace": "system","name": "$DISTRIBUTED_CLOUD_APP_STACK_SITE"}},"inside_network": {}},"labels": {}}],"no_tls": {},"port": 8787,"same_as_endpoint_port": {},"healthcheck": [],"loadbalancer_algorithm": "LB_OVERRIDE","endpoint_selection": "LOCAL_PREFERRED","advanced_options": null}}'
 ```
 or [use the UI](https://docs.cloud.f5.com/docs/how-to/app-networking/origin-pools)
-5. Create an HTTP Load Balancer, including header injection of Portkey provider and credentials
+
+6. Create an HTTP Load Balancer, including header injection of Portkey provider and credentials
 ```shell
 curl --request POST \
   --url https://$DISTRIBUTED_CLOUD_TENANT.console.ves.volterra.io/api/config/namespaces/$DISTRIBUTED_CLOUD_NAMESPACE/http_loadbalancers \
@@ -103,7 +138,8 @@ curl --request POST \
   --data '{"metadata": {"name": "$DISTRIBUTED_CLOUD_SERVICE_NAME","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","labels": {},"annotations": {},"description": "","disable": false},"spec": {"domains": ["$PORTKEY_GATEWAY_FQDN"],"https_auto_cert": {"http_redirect": true,"add_hsts": false,"tls_config": {"default_security": {}},"no_mtls": {},"default_header": {},"enable_path_normalize": {},"port": 443,"non_default_loadbalancer": {},"header_transformation_type": {"default_header_transformation": {}},"connection_idle_timeout": 120000,"http_protocol_options": {"http_protocol_enable_v1_v2": {}}},"advertise_on_public_default_vip": {},"default_route_pools": [{"pool": {"tenant": "$DISTRIBUTED_CLOUD_TENANT_ID","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","name": "$DISTRIBUTED_CLOUD_SERVICE_NAME"},"weight": 1,"priority": 1,"endpoint_subsets": {}}],"origin_server_subset_rule_list": null,"routes": [],"cors_policy": null,"disable_waf": {},"add_location": true,"no_challenge": {},"more_option": {"request_headers_to_add": [{"name": "x-portkey-provider","value": "$PORTKEY_PROVIDER","append": false},{"name": "Authorization","value": "Bearer $PORTKEY_PROVIDER_AUTH_TOKEN","append": false}],"request_headers_to_remove": [],"response_headers_to_add": [],"response_headers_to_remove": [],"max_request_header_size": 60,"buffer_policy": null,"compression_params": null,"custom_errors": {},"javascript_info": null,"jwt": [],"idle_timeout": 30000,"disable_default_error_pages": false,"cookies_to_modify": []},"user_id_client_ip": {},"disable_rate_limit": {},"malicious_user_mitigation": null,"waf_exclusion_rules": [],"data_guard_rules": [],"blocked_clients": [],"trusted_clients": [],"api_protection_rules": null,"ddos_mitigation_rules": [],"service_policies_from_namespace": {},"round_robin": {},"disable_trust_client_ip_headers": {},"disable_ddos_detection": {},"disable_malicious_user_detection": {},"disable_api_discovery": {},"disable_bot_defense": {},"disable_api_definition": {},"disable_ip_reputation": {},"disable_client_side_defense": {},"csrf_policy": null,"graphql_rules": [],"protected_cookies": [],"host_name": "","dns_info": [],"internet_vip_info": [],"system_default_timeouts": {},"jwt_validation": null,"disable_threat_intelligence": {},"l7_ddos_action_default": {},}}'
 ```
 or [use the UI](https://docs.cloud.f5.com/docs/how-to/app-networking/http-load-balancer)
-6. Test the service
+
+7. Test the service
 ```shell
 curl --request POST \
   --url https://$PORTKEY_GATEWAY_FQDN/v1/chat/completions \
@@ -137,6 +173,7 @@ in addition to the response headers, you should get a response body like
 }
 ```
 
+---
 
 ### Cloudflare Workers
 
@@ -159,33 +196,10 @@ npm install
 npm run deploy
 ```
 
+For more details, refer to [Cloudflare Workers official](https://developers.cloudflare.com/workers/).
 <br>
 
-### NodeJS Server
-
-1. Clone the Repository
-
-```sh
-git clone https://github.com/portkey-ai/gateway
-```
-
-2. Install the NPM Dependencies
-
-```sh
-cd gateway
-npm i
-npm run build
-```
-
-<br>
-
-3. Run the Server
-
-```sh
-node build/start-server.js
-```
-
-<br>
+---
 
 ### Docker
 
@@ -199,6 +213,8 @@ For more information on the Docker image, check [here](https://hub.docker.com/r/
 
 <br>
 
+--- 
+
 ### Docker Compose
 
 1. Download Compose File from the Repository:
@@ -214,8 +230,10 @@ docker compose up -d
 ```
 > The service is now running and listening on port 8787
 
+For more details, refer to [Docker Compose official](https://docs.docker.com/compose/).
 <br>
 
+
 ### AWS EC2
 1. Copy the AWS CloudFormation template from below:
 ```yaml
@@ -336,48 +354,123 @@ Outputs:
 
 5. Once the stack is created, you can access the Portkey Gateway URL from the Outputs section.
 
+---
+
+
 ### Replit
 
 [![Deploy on Replit](https://replit.com/badge?caption=Deploy%20on%20Replit)](https://replit.com/@portkey/AI-Gateway?v=1)
 
-<br>
+---
 
 ### Zeabur
 
 [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/templates/RU38E3)
 
-<br>
+---
 
-### Vercel
+### Supabase Functions
+
+1. Clone the Repository:
+   
+```sh
+git clone https://github.com/portkey-ai/gateway
+```
 
-Docs to be written, please help!
+2. Set up a Supabase Account and create a new project (in browser).
+3. Install the Supabase CLI, login using `supabase login
+` and Initialize, follow [guides](https://supabase.com/docs/guides/local-development)
+4. Write Your Supabase Function in:
+```sh
+supabase/functions/your-function-file/index.ts
+```
+5. Deploy the Function (for windows add `npx` in start):
+```sh
+supabase functions deploy your-function-file
+```
+6. Now, its ready to Test, Monitor and Manage Functions (Using url from supabase dashboard):
+   - Similar to `https://your-project-id.supabase.co/functions/v1/your-function-file`
+
+Refer to [Supabase's official documentation](https://supabase.com/docs) for more details.
 
 <br>
 
+---
+
 ### Fastly
 
-Docs to be written, please help!
+1. Create / login to your fastly account (in browser).
+
+2. Clone the Gateway Repository:
+
+```sh
+git clone https://github.com/portkey-ai/gateway
+```
+
+3. Install Fastly’s CLI:
+
+```sh
+npm install fastly-cli
+```
+
+4. Log in to Fastly:
+```sh
+fastly login
+```
+
+5. Create a New Fastly Service:
+```sh
+fastly compute init
+```
+
+6. Write Your Edge Function Code in `src/main.rs` or `src/index.js`
+
+7. Set up the token locally (token is located on `Account > User(left bottom most) > API Token`)
+```sh
+fastly profile create
+```
+Then follow the steps there to add token.
+
+8. Deploy to Fastly
+```sh
+fastly compute publish
+```
+9. Now, its ready to Test, Monitor and Manage Functions (Using url from fastly dashboard):
+    - Similar to `your-new-project.edgecompute.app`
+   
+For more details, refer to [Fastly’s official](https://www.fastly.com/products/edge-compute).
 
 <br>
 
-### AWS Lambda
+---
 
-Docs to be written, please help!
+### Vercel
+
+- Docs to be written, please help!
+- Make sure you've deployed and tested the same.
 
 <br>
 
-### Lambda@edge
+---
+
+### AWS Lambda
 
-Docs to be written, please help!
+- Docs to be written, please help!
+- Make sure you've deployed and tested the same.
 
 <br>
 
-### Supabase Functions
+---
+
+### Lambda edge
 
-Docs to be written, please help!
+- Docs to be written, please help!
+- Make sure you've deployed and tested the same.
 
 <br>
 
+---
+
 ## Enterprise Deployment
 Make your AI app more <ins>reliable</ins> and <ins>forward compatible</ins>, while ensuring complete <ins>data security</ins> and <ins>privacy</ins>.
 

docs/installation-deployments.md

@@ -595,14 +595,14 @@
       }
     },
     {
-      "name": "Model whitelisting",
+      "name": "Allowed Models",
       "id": "modelwhitelist",
       "type": "guardrail",
       "supportedHooks": ["beforeRequestHook"],
       "description": [
         {
           "type": "subHeading",
-          "text": "Check if the model in the request is part of the allowed model list."
+          "text": "Blocks any request whose model isn’t on this list."
         }
       ],
       "parameters": {
@@ -614,7 +614,7 @@
             "description": [
               {
                 "type": "subHeading",
-                "text": "Enter the allowed models."
+                "text": "Enter the allowed models. e.g. gpt-4o, llama-3-70b, mixtral-8x7b"
               }
             ],
             "items": {
@@ -627,7 +627,7 @@
             "description": [
               {
                 "type": "subHeading",
-                "text": "If true, the verdict will be true when model is not in the list"
+                "text": "When on, any model in the list is blocked instead of allowed."
               }
             ],
             "default": false
@@ -706,6 +706,49 @@
         },
         "required": ["jwksUri", "headerKey"]
       }
+    },
+    {
+      "name": "Required Metadata Keys",
+      "id": "requiredMetadataKeys",
+      "type": "guardrail",
+      "supportedHooks": ["beforeRequestHook"],
+      "description": [
+        {
+          "type": "subHeading",
+          "text": "Checks if the metadata contains all the required keys"
+        }
+      ],
+      "parameters": {
+        "type": "object",
+        "properties": {
+          "metadataKeys": {
+            "type": "array",
+            "label": "Metadata Keys",
+            "description": [
+              {
+                "type": "subHeading",
+                "text": "Enter the metadata keys to check for."
+              }
+            ],
+            "items": {
+              "type": "string"
+            }
+          },
+          "operator": {
+            "type": "string",
+            "label": "Operator",
+            "description": [
+              {
+                "type": "subHeading",
+                "text": "Select the operator to use."
+              }
+            ],
+            "enum": ["all", "any", "none"],
+            "default": "all"
+          }
+        },
+        "required": ["metadataKeys", "operator"]
+      }
     }
   ]
 }