Skip to content

Commit bcfb9fe

Browse files
stainless-botRobertCraigie
stainless-bot
authored and
RobertCraigie
committed
chore(client)!: document proxy use + clean up old code
chore: unknown commit message
1 parent e84ad1c commit bcfb9fe

File tree

8 files changed

+165
-53
lines changed

8 files changed

+165
-53
lines changed

‎README.md‎

Lines changed: 47 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -599,25 +599,61 @@ const client = new OpenAI({
599599
Note that if given a `OPENAI_LOG=debug` environment variable, this library will log all requests and responses automatically.
600600
This is intended for debugging purposes only and may change in the future without notice.
601601

602-
### Configuring an HTTP(S) Agent (e.g., for proxies)
602+
### Fetch options
603603

604-
By default, this library uses a stable agent for all http/https requests to reuse TCP connections, eliminating many TCP & TLS handshakes and shaving around 100ms off most requests.
604+
If you want to set custom `fetch` options without overriding the `fetch` function, you can provide a `fetchOptions` object when instantiating the client or making a request. (Request-specific options override client options.)
605605

606-
If you would like to disable or customize this behavior, for example to use the API behind a proxy, you can pass an `httpAgent` which is used for all requests (be they http or https), for example:
606+
```ts
607+
import OpenAI from 'openai';
608+
609+
const client = new OpenAI({
610+
fetchOptions: {
611+
// `RequestInit` options
612+
},
613+
});
614+
```
615+
616+
#### Configuring proxies
617+
618+
To modify proxy behavior, you can provide custom `fetchOptions` that add runtime-specific proxy
619+
options to requests:
620+
621+
<img src="https://raw.githubusercontent.com/stainless-api/sdk-assets/refs/heads/main/node.svg" align="top" width="18" height="21"> **Node** <sup>[[docs](https://github.com/nodejs/undici/blob/main/docs/docs/api/ProxyAgent.md#example---proxyagent-with-fetch)]</sup>
607622

608-
<!-- prettier-ignore -->
609623
```ts
610-
import http from 'http';
611-
import { HttpsProxyAgent } from 'https-proxy-agent';
624+
import OpenAI from 'openai';
625+
import * as undici from 'undici';
626+
627+
const proxyAgent = new undici.ProxyAgent('http://localhost:8888');
628+
const client = new OpenAI({
629+
fetchOptions: {
630+
dispatcher: proxyAgent,
631+
},
632+
});
633+
```
634+
635+
<img src="https://raw.githubusercontent.com/stainless-api/sdk-assets/refs/heads/main/bun.svg" align="top" width="18" height="21"> **Bun** <sup>[[docs](https://bun.sh/guides/http/proxy)]</sup>
636+
637+
```ts
638+
import OpenAI from 'openai';
612639

613-
// Configure the default for all requests:
614640
const client = new OpenAI({
615-
httpAgent: new HttpsProxyAgent(process.env.PROXY_URL),
641+
fetchOptions: {
642+
proxy: 'http://localhost:8888',
643+
},
616644
});
645+
```
617646

618-
// Override per-request:
619-
await client.models.list({
620-
httpAgent: new http.Agent({ keepAlive: false }),
647+
<img src="https://raw.githubusercontent.com/stainless-api/sdk-assets/refs/heads/main/deno.svg" align="top" width="18" height="21"> **Deno** <sup>[[docs](https://docs.deno.com/api/deno/~/Deno.createHttpClient)]</sup>
648+
649+
```ts
650+
import OpenAI from 'npm:openai';
651+
652+
const httpClient = Deno.createHttpClient({ proxy: { url: 'http://localhost:8888' } });
653+
const client = new OpenAI({
654+
fetchOptions: {
655+
client: httpClient,
656+
},
621657
});
622658
```
623659

‎scripts/utils/attw-report.cjs‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,10 @@ const problems = Object.values(JSON.parse(fs.readFileSync('.attw.json', 'utf-8')
88
(
99
(problem.kind === 'CJSResolvesToESM' && problem.entrypoint.endsWith('.mjs')) ||
1010
// This is intentional for backwards compat reasons.
11-
(problem.kind === 'MissingExportEquals' && problem.implementationFileName.endsWith('/index.js'))
11+
(problem.kind === 'MissingExportEquals' && problem.implementationFileName.endsWith('/index.js')) ||
12+
// this is intentional, we deliberately attempt to import types that may not exist from parent node_modules
13+
// folders to better support various runtimes without triggering automatic type acquisition.
14+
(problem.kind === 'InternalResolutionError' && problem.moduleSpecifier.includes('node_modules'))
1215
)
1316
),
1417
);

‎src/client.ts‎

Lines changed: 9 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
22

33
import type { RequestInit, RequestInfo, BodyInit } from './internal/builtin-types';
4-
import type { HTTPMethod, PromiseOrValue } from './internal/types';
4+
import type { HTTPMethod, PromiseOrValue, MergedRequestInit } from './internal/types';
55
import { uuid4 } from './internal/utils/uuid';
66
import { validatePositiveInteger, isAbsoluteURL } from './internal/utils/values';
77
import { sleep } from './internal/utils/sleep';
@@ -183,14 +183,11 @@ export interface ClientOptions {
183183
* much longer than this timeout before the promise succeeds or fails.
184184
*/
185185
timeout?: number | undefined;
186-
187186
/**
188-
* An HTTP agent used to manage HTTP(S) connections.
189-
*
190-
* If not provided, an agent will be constructed by default in the Node.js environment,
191-
* otherwise no agent is used.
187+
* Additional `RequestInit` options to be passed to `fetch` calls.
188+
* Properties will be overridden by per-request `fetchOptions`.
192189
*/
193-
httpAgent?: Shims.Agent | undefined;
190+
fetchOptions?: MergedRequestInit | undefined;
194191

195192
/**
196193
* Specify a custom `fetch` function implementation.
@@ -259,7 +256,7 @@ export class OpenAI {
259256
timeout: number;
260257
logger: Logger | undefined;
261258
logLevel: LogLevel | undefined;
262-
httpAgent: Shims.Agent | undefined;
259+
fetchOptions: MergedRequestInit | undefined;
263260

264261
private fetch: Fetch;
265262
#encoder: Opts.RequestEncoder;
@@ -274,7 +271,7 @@ export class OpenAI {
274271
* @param {string | null | undefined} [opts.project=process.env['OPENAI_PROJECT_ID'] ?? null]
275272
* @param {string} [opts.baseURL=process.env['OPENAI_BASE_URL'] ?? https://api.openai.com/v1] - Override the default base URL for the API.
276273
* @param {number} [opts.timeout=10 minutes] - The maximum amount of time (in milliseconds) the client will wait for a response before timing out.
277-
* @param {number} [opts.httpAgent] - An HTTP agent used to manage HTTP(s) connections.
274+
* @param {MergedRequestInit} [opts.fetchOptions] - Additional `RequestInit` options to be passed to `fetch` calls.
278275
* @param {Fetch} [opts.fetch] - Specify a custom `fetch` function implementation.
279276
* @param {number} [opts.maxRetries=2] - The maximum number of times the client will retry a request.
280277
* @param {HeadersLike} opts.defaultHeaders - Default headers to include with every request to the API.
@@ -319,7 +316,7 @@ export class OpenAI {
319316
this.logLevel = envLevel;
320317
}
321318
}
322-
this.httpAgent = options.httpAgent;
319+
this.fetchOptions = options.fetchOptions;
323320
this.maxRetries = options.maxRetries ?? 2;
324321
this.fetch = options.fetch ?? Shims.getDefaultFetch();
325322
this.#encoder = Opts.FallbackEncoder;
@@ -657,30 +654,18 @@ export class OpenAI {
657654
const url = this.buildURL(path!, query as Record<string, unknown>);
658655
if ('timeout' in options) validatePositiveInteger('timeout', options.timeout);
659656
const timeout = options.timeout ?? this.timeout;
660-
const httpAgent = options.httpAgent ?? this.httpAgent;
661-
const minAgentTimeout = timeout + 1000;
662-
if (
663-
typeof (httpAgent as any)?.options?.timeout === 'number' &&
664-
minAgentTimeout > ((httpAgent as any).options.timeout ?? 0)
665-
) {
666-
// Allow any given request to bump our agent active socket timeout.
667-
// This may seem strange, but leaking active sockets should be rare and not particularly problematic,
668-
// and without mutating agent we would need to create more of them.
669-
// This tradeoff optimizes for performance.
670-
(httpAgent as any).options.timeout = minAgentTimeout;
671-
}
672-
673657
const { bodyHeaders, body } = this.buildBody({ options });
674658
const reqHeaders = this.buildHeaders({ options, method, bodyHeaders, retryCount });
675659

676660
const req: FinalizedRequestInit = {
677661
method,
678662
headers: reqHeaders,
679-
...(httpAgent && { agent: httpAgent }),
680663
...(options.signal && { signal: options.signal }),
681664
...((globalThis as any).ReadableStream &&
682665
body instanceof (globalThis as any).ReadableStream && { duplex: 'half' }),
683666
...(body && { body }),
667+
...((this.fetchOptions as any) ?? {}),
668+
...((options.fetchOptions as any) ?? {}),
684669
};
685670

686671
return { req, url, timeout };

‎src/internal/builtin-types.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
22

3-
export type Fetch = typeof fetch;
3+
export type Fetch = (input: string | URL | Request, init?: RequestInit) => Promise<Response>;
44

55
/**
66
* An alias to the builtin `RequestInit` type so we can

‎src/internal/request-options.ts‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,10 @@
22

33
import { NullableHeaders } from './headers';
44

5-
import type { Agent } from './shims';
65
import type { BodyInit } from './builtin-types';
76
import { isEmptyObj, hasOwn } from './utils/values';
87
import { Stream } from '../streaming';
9-
import type { HTTPMethod, KeysEnum } from './types';
8+
import type { HTTPMethod, KeysEnum, MergedRequestInit } from './types';
109
import { type HeadersLike } from './headers';
1110

1211
export type FinalRequestOptions = RequestOptions & { method: HTTPMethod; path: string };
@@ -20,7 +19,7 @@ export type RequestOptions = {
2019
maxRetries?: number;
2120
stream?: boolean | undefined;
2221
timeout?: number;
23-
httpAgent?: Agent;
22+
fetchOptions?: MergedRequestInit;
2423
signal?: AbortSignal | undefined | null;
2524
idempotencyKey?: string;
2625

@@ -41,7 +40,7 @@ const requestOptionsKeys: KeysEnum<RequestOptions> = {
4140
maxRetries: true,
4241
stream: true,
4342
timeout: true,
44-
httpAgent: true,
43+
fetchOptions: true,
4544
signal: true,
4645
idempotencyKey: true,
4746

‎src/internal/shims.ts‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -10,18 +10,6 @@
1010
import { type Fetch } from './builtin-types';
1111
import { type ReadableStream } from './shim-types';
1212

13-
/**
14-
* A minimal copy of the `Agent` type from `undici-types` so we can
15-
* use it in the `ClientOptions` type.
16-
*
17-
* https://nodejs.org/api/http.html#class-httpagent
18-
*/
19-
export interface Agent {
20-
dispatch(options: any, handler: any): boolean;
21-
closed: boolean;
22-
destroyed: boolean;
23-
}
24-
2513
export function getDefaultFetch(): Fetch {
2614
if (typeof fetch !== 'undefined') {
2715
return fetch;

‎src/internal/types.ts‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,95 @@ export type PromiseOrValue<T> = T | Promise<T>;
44
export type HTTPMethod = 'get' | 'post' | 'put' | 'patch' | 'delete';
55

66
export type KeysEnum<T> = { [P in keyof Required<T>]: true };
7+
8+
type NotAny<T> = [unknown] extends [T] ? never : T;
9+
type Literal<T> = PropertyKey extends T ? never : T;
10+
type MappedLiteralKeys<T> = T extends any ? Literal<keyof T> : never;
11+
type MappedIndex<T, K> =
12+
T extends any ?
13+
K extends keyof T ?
14+
T[K]
15+
: never
16+
: never;
17+
18+
/**
19+
* Some environments overload the global fetch function, and Parameters<T> only gets the last signature.
20+
*/
21+
type OverloadedParameters<T> =
22+
T extends (
23+
{
24+
(...args: infer A): unknown;
25+
(...args: infer B): unknown;
26+
(...args: infer C): unknown;
27+
(...args: infer D): unknown;
28+
}
29+
) ?
30+
A | B | C | D
31+
: T extends (
32+
{
33+
(...args: infer A): unknown;
34+
(...args: infer B): unknown;
35+
(...args: infer C): unknown;
36+
}
37+
) ?
38+
A | B | C
39+
: T extends (
40+
{
41+
(...args: infer A): unknown;
42+
(...args: infer B): unknown;
43+
}
44+
) ?
45+
A | B
46+
: T extends (...args: infer A) => unknown ? A
47+
: never;
48+
49+
/* eslint-disable */
50+
/**
51+
* These imports attempt to get types from a parent package's dependencies.
52+
* Unresolved bare specifiers can trigger [automatic type acquisition][1] in some projects, which
53+
* would cause typescript to show types not present at runtime. To avoid this, we import
54+
* directly from parent node_modules folders.
55+
*
56+
* We need to check multiple levels because we don't know what directory structure we'll be in.
57+
* For example, pnpm generates directories like this:
58+
* ```
59+
* node_modules
60+
* ├── .pnpm
61+
* │ └── [email protected]
62+
* │ └── node_modules
63+
* │ └── pkg
64+
* │ └── internal
65+
* │ └── types.d.ts
66+
* ├── pkg -> .pnpm/[email protected]/node_modules/pkg
67+
* └── undici
68+
* ```
69+
*
70+
* [1]: https://www.typescriptlang.org/tsconfig/#typeAcquisition
71+
*/
72+
/** @ts-ignore For users with \@types/node */
73+
type UndiciTypesRequestInit = NotAny<import('../node_modules/undici-types').RequestInit> | NotAny<import('../../node_modules/undici-types').RequestInit> | NotAny<import('../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../../../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../../../../../../node_modules/undici-types').RequestInit> | NotAny<import('../../../../../../../../../../node_modules/undici-types').RequestInit>;
74+
/** @ts-ignore For users with undici */
75+
type UndiciRequestInit = NotAny<import('../node_modules/undici').RequestInit> | NotAny<import('../../node_modules/undici').RequestInit> | NotAny<import('../../../node_modules/undici').RequestInit> | NotAny<import('../../../../node_modules/undici').RequestInit> | NotAny<import('../../../../../node_modules/undici').RequestInit> | NotAny<import('../../../../../../node_modules/undici').RequestInit> | NotAny<import('../../../../../../../node_modules/undici').RequestInit> | NotAny<import('../../../../../../../../node_modules/undici').RequestInit> | NotAny<import('../../../../../../../../../node_modules/undici').RequestInit> | NotAny<import('../../../../../../../../../../node_modules/undici').RequestInit>;
76+
/** @ts-ignore For users with \@types/bun */
77+
type BunRequestInit = globalThis.FetchRequestInit;
78+
/** @ts-ignore For users with node-fetch */
79+
type NodeFetchRequestInit = NotAny<import('../node_modules/node-fetch').RequestInit> | NotAny<import('../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../../../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../../../../../../node_modules/node-fetch').RequestInit> | NotAny<import('../../../../../../../../../../node_modules/node-fetch').RequestInit>;
80+
/** @ts-ignore For users who use Deno */
81+
type FetchRequestInit = NonNullable<OverloadedParameters<typeof fetch>[1]>;
82+
/* eslint-enable */
83+
84+
type RequestInits =
85+
| NotAny<UndiciTypesRequestInit>
86+
| NotAny<UndiciRequestInit>
87+
| NotAny<BunRequestInit>
88+
| NotAny<NodeFetchRequestInit>
89+
| NotAny<RequestInit>
90+
| NotAny<FetchRequestInit>;
91+
92+
/**
93+
* This type contains `RequestInit` options that may be available on the current runtime,
94+
* including per-platform extensions like `dispatcher`, `agent`, `client`, etc.
95+
*/
96+
export type MergedRequestInit = {
97+
[K in MappedLiteralKeys<RequestInits>]?: MappedIndex<RequestInits, K> | undefined;
98+
};

‎tests/index.test.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,15 @@ describe('instantiate client', () => {
179179
expect(response).toEqual({ url: 'http://localhost:5000/foo', custom: true });
180180
});
181181

182+
test('explicit global fetch', async () => {
183+
// make sure the global fetch type is assignable to our Fetch type
184+
const client = new OpenAI({
185+
baseURL: 'http://localhost:5000/',
186+
apiKey: 'My API Key',
187+
fetch: defaultFetch,
188+
});
189+
});
190+
182191
test('custom signal', async () => {
183192
const client = new OpenAI({
184193
baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',

0 commit comments

Comments
 (0)