Skip to content

feat: add docs for uploading file to vertex for inference purposes #440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 129 additions & 1 deletion integrations/llms/vertex-ai/files.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -304,4 +304,132 @@ print(file_content)
</Tabs>


Note: The `ListFiles` endpoint is not supported for Vertex AI.
Note: The `ListFiles` endpoint is not supported for Vertex AI.


### Uploading Files for Inference

Vertex AI supports inference with files that are uploaded to GCS, to upload files for inference i.e not for `batch` and `fine-tune`, please check below code snippet.

<Tabs>
<Tab title="Python">
```python
from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
api_key="PORTKEY_API_KEY", # Replace with your Portkey API key
provider="@VERTEX_PROVIDER",
vertex_storage_bucket_name="your_bucket_name", # Specify the GCS bucket name
provider_file_name="your_file_name.png", # Specify the file name in GCS
provider_model="gemini-1.5-flash-001" # Specify the model to use
)

upload_file_response = portkey.files.create(
purpose="upload",
file=open("your_file_name.png", "rb"),
extra_headers={"x-portkey-file-purpose": "upload"}
)

print(upload_file_response)
```
</Tab>
<Tab title="NodeJS">
```js
import { Portkey } from 'portkey-ai';
import * as fs from 'fs';

// Initialize the Portkey client
const portkey = Portkey({
apiKey: "PORTKEY_API_KEY", // Replace with your Portkey API key
provider:"@VERTEX_PROVIDER",
vertexStorageBucketName: "your_bucket_name", // Specify the GCS bucket name
providerFileName: "your_file_name.png", // Specify the file name in GCS
providerModel: "gemini-1.5-flash-001" // Specify the model to use
});

const uploadFile = async () => {
const file = await portkey.files.create({
purpose: "upload",
file: fs.createReadStream("your_file_name.png"),
extra_headers: {"x-portkey-file-purpose": "upload"}
});

console.log(file);
}

uploadFile();
```
</Tab>
<Tab title="REST">
```sh
curl -X POST --header 'x-portkey-api-key: <portkey_api_key>' \
--header 'x-portkey-provider: <vertex_virtual_key>' \
--header 'x-portkey-vertex-storage-bucket-name: <bucket_name>' \
--header 'x-portkey-provider-file-name: <file_name>.png' \
--header 'x-portkey-provider-model: <model_name>' \
--header 'x-portkey-file-purpose: "upload"' \
--form 'purpose="upload"' \
--form 'file=@your_file_name.png' \
'https://api.portkey.ai/v1/files'
```
</Tab>
<Tab title="OpenAI NodeJS">
```js
import OpenAI from 'openai';
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai';
import * as fs from 'fs';

const openai = new OpenAI({
apiKey: 'OPENAI_API_KEY',
baseURL: PORTKEY_GATEWAY_URL,
defaultHeaders: createHeaders({
provider:"@VERTEX_PROVIDER",
apiKey: "PORTKEY_API_KEY",
vertexStorageBucketName: "your_bucket_name",
providerFileName: "your_file_name.png",
providerModel: "gemini-1.5-flash-001"
})
});

const uploadFile = async () => {
const file = await openai.files.create({
purpose: "upload",
file: fs.createReadStream("your_file_name.png"),
extraHeaders: {"x-portkey-file-purpose": "upload"}
});

console.log(file);
}

uploadFile();
```
</Tab>
<Tab title="OpenAI Python">
```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

openai = OpenAI(
api_key='OPENAI_API_KEY',
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
provider="@VERTEX_PROVIDER",
api_key="PORTKEY_API_KEY",
vertex_storage_bucket_name="your_bucket_name",
provider_file_name="your_file_name.png",
provider_model="gemini-1.5-flash-001"
)
)

upload_file_response = openai.files.create(
purpose="upload",
file=open("your_file_name.png", "rb"),
extra_headers={"x-portkey-file-purpose": "upload"}
)

print(upload_file_response)
```
</Tab>

</Tabs>