diff --git a/docs-devsite/ai.md b/docs-devsite/ai.md index c6e8fa365f7..ce69048092e 100644 --- a/docs-devsite/ai.md +++ b/docs-devsite/ai.md @@ -149,7 +149,7 @@ The Firebase AI Web SDK. | Variable | Description | | --- | --- | | [AIErrorCode](./ai.md#aierrorcode) | Standardized error codes that [AIError](./ai.aierror.md#aierror_class) can have. | -| [BackendType](./ai.md#backendtype) | An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with.These values are assigned to the backendType property within the specific backend configuration objects ([GoogleAIBackend](./ai.googleaibackend.md#googleaibackend_class) or [VertexAIBackend](./ai.vertexaibackend.md#vertexaibackend_class)) to identify which service to target. | +| [BackendType](./ai.md#backendtype) | An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. | | [BlockReason](./ai.md#blockreason) | Reason that a prompt was blocked. | | [FinishReason](./ai.md#finishreason) | Reason that a candidate finished. | | [FunctionCallingMode](./ai.md#functioncallingmode) | | @@ -158,9 +158,9 @@ The Firebase AI Web SDK. | [HarmCategory](./ai.md#harmcategory) | Harm categories that would cause prompts or candidates to be blocked. | | [HarmProbability](./ai.md#harmprobability) | Probability that a prompt or candidate matches a harm category. | | [HarmSeverity](./ai.md#harmseverity) | Harm severity levels. | -| [ImagenAspectRatio](./ai.md#imagenaspectratio) | (Public Preview) Aspect ratios for Imagen images.To specify an aspect ratio for generated images, set the aspectRatio property in your [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface).See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) for more details and examples of the supported aspect ratios. | -| [ImagenPersonFilterLevel](./ai.md#imagenpersonfilterlevel) | (Public Preview) A filter level controlling whether generation of images containing people or faces is allowed.See the personGeneration documentation for more details. | -| [ImagenSafetyFilterLevel](./ai.md#imagensafetyfilterlevel) | (Public Preview) A filter level controlling how aggressively to filter sensitive content.Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, violence, sexual, derogatory, and toxic). This filter level controls how aggressively to filter out potentially harmful content from responses. See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) and the [Responsible AI and usage guidelines](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#safety-filters) for more details. | +| [ImagenAspectRatio](./ai.md#imagenaspectratio) | (Public Preview) Aspect ratios for Imagen images. | +| [ImagenPersonFilterLevel](./ai.md#imagenpersonfilterlevel) | (Public Preview) A filter level controlling whether generation of images containing people or faces is allowed. | +| [ImagenSafetyFilterLevel](./ai.md#imagensafetyfilterlevel) | (Public Preview) A filter level controlling how aggressively to filter sensitive content. | | [InferenceMode](./ai.md#inferencemode) | (Public Preview) Determines whether inference happens on-device or in-cloud. | | [Language](./ai.md#language) | (Public Preview) The programming language of the code. | | [LiveResponseType](./ai.md#liveresponsetype) | (Public Preview) The types of responses that can be returned by [LiveSession.receive()](./ai.livesession.md#livesessionreceive). | @@ -176,7 +176,7 @@ The Firebase AI Web SDK. | Type Alias | Description | | --- | --- | | [AIErrorCode](./ai.md#aierrorcode) | Standardized error codes that [AIError](./ai.aierror.md#aierror_class) can have. | -| [BackendType](./ai.md#backendtype) | Type alias representing valid backend types. It can be either 'VERTEX_AI' or 'GOOGLE_AI'. | +| [BackendType](./ai.md#backendtype) | An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. | | [BlockReason](./ai.md#blockreason) | Reason that a prompt was blocked. | | [FinishReason](./ai.md#finishreason) | Reason that a candidate finished. | | [FunctionCallingMode](./ai.md#functioncallingmode) | | @@ -185,15 +185,15 @@ The Firebase AI Web SDK. | [HarmCategory](./ai.md#harmcategory) | Harm categories that would cause prompts or candidates to be blocked. | | [HarmProbability](./ai.md#harmprobability) | Probability that a prompt or candidate matches a harm category. | | [HarmSeverity](./ai.md#harmseverity) | Harm severity levels. | -| [ImagenAspectRatio](./ai.md#imagenaspectratio) | (Public Preview) Aspect ratios for Imagen images.To specify an aspect ratio for generated images, set the aspectRatio property in your [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface).See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) for more details and examples of the supported aspect ratios. | -| [ImagenPersonFilterLevel](./ai.md#imagenpersonfilterlevel) | (Public Preview) A filter level controlling whether generation of images containing people or faces is allowed.See the personGeneration documentation for more details. | -| [ImagenSafetyFilterLevel](./ai.md#imagensafetyfilterlevel) | (Public Preview) A filter level controlling how aggressively to filter sensitive content.Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, violence, sexual, derogatory, and toxic). This filter level controls how aggressively to filter out potentially harmful content from responses. See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) and the [Responsible AI and usage guidelines](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#safety-filters) for more details. | +| [ImagenAspectRatio](./ai.md#imagenaspectratio) | (Public Preview) Aspect ratios for Imagen images. | +| [ImagenPersonFilterLevel](./ai.md#imagenpersonfilterlevel) | (Public Preview) A filter level controlling whether generation of images containing people or faces is allowed. | +| [ImagenSafetyFilterLevel](./ai.md#imagensafetyfilterlevel) | (Public Preview) A filter level controlling how aggressively to filter sensitive content. | | [InferenceMode](./ai.md#inferencemode) | (Public Preview) Determines whether inference happens on-device or in-cloud. | | [Language](./ai.md#language) | (Public Preview) The programming language of the code. | | [LanguageModelMessageContentValue](./ai.md#languagemodelmessagecontentvalue) | (Public Preview) Content formats that can be provided as on-device message content. | | [LanguageModelMessageRole](./ai.md#languagemodelmessagerole) | (Public Preview) Allowable roles for on-device language model usage. | | [LanguageModelMessageType](./ai.md#languagemodelmessagetype) | (Public Preview) Allowable types for on-device language model messages. | -| [LiveResponseType](./ai.md#liveresponsetype) | (Public Preview) The types of responses that can be returned by [LiveSession.receive()](./ai.livesession.md#livesessionreceive). This is a property on all messages that can be used for type narrowing. This property is not returned by the server, it is assigned to a server message object once it's parsed. | +| [LiveResponseType](./ai.md#liveresponsetype) | (Public Preview) The types of responses that can be returned by [LiveSession.receive()](./ai.livesession.md#livesessionreceive). | | [Modality](./ai.md#modality) | Content part modality. | | [Outcome](./ai.md#outcome) | (Public Preview) Represents the result of the code execution. | | [Part](./ai.md#part) | Content part - includes text, image/video, or function call/response part types. | @@ -437,9 +437,9 @@ AIErrorCode: { ## BackendType -An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with. +An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. -These values are assigned to the `backendType` property within the specific backend configuration objects ([GoogleAIBackend](./ai.googleaibackend.md#googleaibackend_class) or [VertexAIBackend](./ai.vertexaibackend.md#vertexaibackend_class)) to identify which service to target. +This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with. These values are assigned to the `backendType` property within the specific backend configuration objects ([GoogleAIBackend](./ai.googleaibackend.md#googleaibackend_class) or [VertexAIBackend](./ai.vertexaibackend.md#vertexaibackend_class)) to identify which service to target.
VERTEX\_AI: Identifies the backend service for the Vertex AI Gemini API provided through Google Cloud. Use this constant when creating a [VertexAIBackend](./ai.vertexaibackend.md#vertexaibackend_class) configuration.
GOOGLE\_AI: Identifies the backend service for the Gemini Developer API ([Google AI](https://ai.google/)). Use this constant when creating a [GoogleAIBackend](./ai.googleaibackend.md#googleaibackend_class) configuration. Signature: @@ -454,6 +454,8 @@ BackendType: { Reason that a prompt was blocked. +SAFETY: Content was blocked by safety settings.
OTHER: Content was blocked, but the reason is uncategorized.
BLOCKLIST: Content was blocked because it contained terms from the terminology blocklist.
PROHIBITED\_CONTENT: Content was blocked due to prohibited content. + Signature: ```typescript @@ -469,6 +471,8 @@ BlockReason: { Reason that a candidate finished. +STOP: Natural stop point of the model or provided stop sequence.
MAX\_TOKENS: The maximum number of tokens as specified in the request was reached.
SAFETY: The candidate content was flagged for safety reasons.
RECITATION: The candidate content was flagged for recitation reasons.
OTHER: Unknown reason.
BLOCKLIST: The candidate content contained forbidden terms.
PROHIBITED\_CONTENT: The candidate content potentially contained prohibited content.
SPII: The candidate content potentially contained Sensitive Personally Identifiable Information (SPII).
MALFORMED\_FUNCTION\_CALL: The function call generated by the model was invalid. + Signature: ```typescript @@ -487,6 +491,7 @@ FinishReason: { ## FunctionCallingMode +- AUTO: Default model behavior; model decides to predict either a function call or a natural language response. - ANY: Model is constrained to always predicting a function call only. If `allowed_function_names` is set, the predicted function call will be limited to any one of `allowed_function_names`, else the predicted function call will be any one of the provided `function_declarations`.
NONE: Model will not predict any function call. Model behavior is same as when not passing any function declarations. Signature: @@ -502,6 +507,8 @@ FunctionCallingMode: { This property is not supported in the Gemini Developer API ([GoogleAIBackend](./ai.googleaibackend.md#googleaibackend_class)). +SEVERITY: The harm block method uses both probability and severity scores.
PROBABILITY: The harm block method uses the probability score. + Signature: ```typescript @@ -515,6 +522,8 @@ HarmBlockMethod: { Threshold above which a prompt or candidate will be blocked. +BLOCK\_LOW\_AND\_ABOVE: Content with `NEGLIGIBLE` will be allowed.
BLOCK\_MEDIUM\_AND\_ABOVE: Content with `NEGLIGIBLE` and `LOW` will be allowed.
BLOCK\_ONLY\_HIGH: Content with `NEGLIGIBLE`, `LOW`, and `MEDIUM` will be allowed.
BLOCK\_NONE: All content will be allowed.
OFF: All content will be allowed. This is the same as `BLOCK_NONE`, but the metadata corresponding to the `HarmCategory` will not be present in the response. + Signature: ```typescript @@ -531,6 +540,8 @@ HarmBlockThreshold: { Harm categories that would cause prompts or candidates to be blocked. +HARM\_CATEGORY\_HATE\_SPEECH: Category for hate speech.
HARM\_CATEGORY\_SEXUALLY\_EXPLICIT: Category for sexually explicit content.
HARM\_CATEGORY\_HARASSMENT: Category for harassment.
HARM\_CATEGORY\_DANGEROUS\_CONTENT: Category for dangerous content. + Signature: ```typescript @@ -546,6 +557,8 @@ HarmCategory: { Probability that a prompt or candidate matches a harm category. +NEGLIGIBLE: Content has a negligible chance of being unsafe.
LOW: Content has a low chance of being unsafe.
MEDIUM: Content has a medium chance of being unsafe.
HIGH: Content has a high chance of being unsafe. + Signature: ```typescript @@ -561,6 +574,8 @@ HarmProbability: { Harm severity levels. +HARM\_SEVERITY\_NEGLIGIBLE: Negligible level of harm severity.
HARM\_SEVERITY\_LOW: Low level of harm severity.
HARM\_SEVERITY\_MEDIUM: Medium level of harm severity.
HARM\_SEVERITY\_HIGH: High level of harm severity.
HARM\_SEVERITY\_UNSUPPORTED: Harm severity is not supported. The GoogleAI backend does not support `HarmSeverity`, so this value is used as a fallback. + Signature: ```typescript @@ -580,9 +595,7 @@ HarmSeverity: { Aspect ratios for Imagen images. -To specify an aspect ratio for generated images, set the `aspectRatio` property in your [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface). - -See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) for more details and examples of the supported aspect ratios. +To specify an aspect ratio for generated images, set the `aspectRatio` property in your [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface). See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) for more details and examples of the supported aspect ratios.
SQUARE: Square (1:1) aspect ratio.
LANDSCAPE\_3x4: Landscape (3:4) aspect ratio.
PORTRAIT\_4x3: Portrait (4:3) aspect ratio.
LANDSCAPE\_16x9: Landscape (16:9) aspect ratio.
PORTRAIT\_9x16: Portrait (9:16) aspect ratio. Signature: @@ -603,7 +616,7 @@ ImagenAspectRatio: { A filter level controlling whether generation of images containing people or faces is allowed. -See the personGeneration documentation for more details. +See the personGeneration documentation for more details.
BLOCK\_ALL: Disallow generation of images containing people or faces; images of people are filtered out.
ALLOW\_ADULT: Allow generation of images containing adults only; images of children are filtered out. Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the [Responsible AI and usage guidelines](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#person-face-gen) for more details.
ALLOW\_ALL: Allow generation of images containing adults only; images of children are filtered out. Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the [Responsible AI and usage guidelines](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#person-face-gen) for more details. Signature: @@ -622,7 +635,7 @@ ImagenPersonFilterLevel: { A filter level controlling how aggressively to filter sensitive content. -Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, `violence`, `sexual`, `derogatory`, and `toxic`). This filter level controls how aggressively to filter out potentially harmful content from responses. See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) and the [Responsible AI and usage guidelines](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#safety-filters) for more details. +Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, `violence`, `sexual`, `derogatory`, and `toxic`). This filter level controls how aggressively to filter out potentially harmful content from responses. See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) and the [Responsible AI and usage guidelines](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#safety-filters) for more details.
BLOCK\_LOW\_AND\_ABOVE: The most aggressive filtering level; most strict blocking.
BLOCK\_MEDIUM\_AND\_ABOVE: Blocks some sensitive prompts and responses.
BLOCK\_ONLY\_HIGH: Blocks few sensitive prompts and responses.
BLOCK\_NONE: The least aggressive filtering level; blocks very few sensitive prompts and responses. Access to this feature is restricted and may require your case to be reviewed and approved by Cloud support. Signature: @@ -678,6 +691,8 @@ Language: { The types of responses that can be returned by [LiveSession.receive()](./ai.livesession.md#livesessionreceive). +SERVER\_CONTENT: An incremental content update from the model.
TOOL\_CALL: A request from the model for the client to execute one or more functions.
TOOL\_CALL\_CANCELLATION: Notification to cancel a previous function call triggered by [LiveServerToolCall](./ai.liveservertoolcall.md#liveservertoolcall_interface). + Signature: ```typescript @@ -692,6 +707,8 @@ LiveResponseType: { Content part modality. +MODALITY\_UNSPECIFIED: Unspecified modality.
TEXT: Plain text.
IMAGE: Image.
VIDEO: Video.
AUDIO: Audio.
DOCUMENT: Document (for example, PDF). + Signature: ```typescript @@ -727,6 +744,8 @@ Outcome: { Possible roles. +user: The user is the one making the request.
model: The model is the one generating the response.
function: The role for a response from a function call.
system: The role for a system instruction. + Signature: ```typescript @@ -740,6 +759,8 @@ POSSIBLE_ROLES: readonly ["user", "model", "function", "system"] Generation modalities to be returned in generation responses. +TEXT: Text.
IMAGE: Image.
AUDIO: Audio. + Signature: ```typescript @@ -800,7 +821,9 @@ export type AIErrorCode = (typeof AIErrorCode)[keyof typeof AIErrorCode]; ## BackendType -Type alias representing valid backend types. It can be either `'VERTEX_AI'` or `'GOOGLE_AI'`. +An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. + +This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with. These values are assigned to the `backendType` property within the specific backend configuration objects ([GoogleAIBackend](./ai.googleaibackend.md#googleaibackend_class) or [VertexAIBackend](./ai.vertexaibackend.md#vertexaibackend_class)) to identify which service to target. Signature: @@ -894,9 +917,7 @@ export type HarmSeverity = (typeof HarmSeverity)[keyof typeof HarmSeverity]; Aspect ratios for Imagen images. -To specify an aspect ratio for generated images, set the `aspectRatio` property in your [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface). - -See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) for more details and examples of the supported aspect ratios. +To specify an aspect ratio for generated images, set the `aspectRatio` property in your [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface). See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images) for more details and examples of the supported aspect ratios. Signature: @@ -1004,7 +1025,7 @@ export type LanguageModelMessageType = 'text' | 'image' | 'audio'; > This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment. > -The types of responses that can be returned by [LiveSession.receive()](./ai.livesession.md#livesessionreceive). This is a property on all messages that can be used for type narrowing. This property is not returned by the server, it is assigned to a server message object once it's parsed. +The types of responses that can be returned by [LiveSession.receive()](./ai.livesession.md#livesessionreceive). Signature: @@ -1062,6 +1083,8 @@ export type ResponseModality = (typeof ResponseModality)[keyof typeof ResponseMo Role is the producer of the content. +user: The user is the one making the request.
model: The model is the one generating the response.
function: The role for a response from a function call.
system: The role for a system instruction. + Signature: ```typescript diff --git a/packages/ai/src/public-types.ts b/packages/ai/src/public-types.ts index fff41251a01..3eb2a152356 100644 --- a/packages/ai/src/public-types.ts +++ b/packages/ai/src/public-types.ts @@ -53,32 +53,35 @@ export interface AI { /** * An enum-like object containing constants that represent the supported backends * for the Firebase AI SDK. - * This determines which backend service (Vertex AI Gemini API or Gemini Developer API) - * the SDK will communicate with. * - * These values are assigned to the `backendType` property within the specific backend - * configuration objects ({@link GoogleAIBackend} or {@link VertexAIBackend}) to identify - * which service to target. + * @remarks + * This determines which backend service (Vertex AI Gemini API or Gemini Developer API) + * the SDK will communicate with. These values are assigned to the `backendType` property + * within the specific backend configuration objects ({@link GoogleAIBackend} or + * {@link VertexAIBackend}) to identify which service to target. + *
+ * VERTEX_AI: Identifies the backend service for the Vertex AI Gemini API provided through Google Cloud. + * Use this constant when creating a {@link VertexAIBackend} configuration. + *
+ * GOOGLE_AI: Identifies the backend service for the Gemini Developer API ({@link https://ai.google/ | Google AI}). + * Use this constant when creating a {@link GoogleAIBackend} configuration. * * @public */ export const BackendType = { - /** - * Identifies the backend service for the Vertex AI Gemini API provided through Google Cloud. - * Use this constant when creating a {@link VertexAIBackend} configuration. - */ VERTEX_AI: 'VERTEX_AI', - - /** - * Identifies the backend service for the Gemini Developer API ({@link https://ai.google/ | Google AI}). - * Use this constant when creating a {@link GoogleAIBackend} configuration. - */ GOOGLE_AI: 'GOOGLE_AI' } as const; // Using 'as const' makes the string values literal types /** - * Type alias representing valid backend types. - * It can be either `'VERTEX_AI'` or `'GOOGLE_AI'`. + * An enum-like object containing constants that represent the supported backends + * for the Firebase AI SDK. + * + * @remarks + * This determines which backend service (Vertex AI Gemini API or Gemini Developer API) + * the SDK will communicate with. These values are assigned to the `backendType` property + * within the specific backend configuration objects ({@link GoogleAIBackend} or + * {@link VertexAIBackend}) to identify which service to target. * * @public */ diff --git a/packages/ai/src/types/enums.ts b/packages/ai/src/types/enums.ts index cd7029df3b0..a0a6ef8cbad 100644 --- a/packages/ai/src/types/enums.ts +++ b/packages/ai/src/types/enums.ts @@ -17,18 +17,48 @@ /** * Role is the producer of the content. + * + * @remarks + * user: The user is the one making the request. + *
+ * model: The model is the one generating the response. + *
+ * function: The role for a response from a function call. + *
+ * system: The role for a system instruction. + * * @public */ export type Role = (typeof POSSIBLE_ROLES)[number]; /** * Possible roles. + * + * @remarks + * user: The user is the one making the request. + *
+ * model: The model is the one generating the response. + *
+ * function: The role for a response from a function call. + *
+ * system: The role for a system instruction. + * * @public */ export const POSSIBLE_ROLES = ['user', 'model', 'function', 'system'] as const; /** * Harm categories that would cause prompts or candidates to be blocked. + * + * @remarks + * HARM_CATEGORY_HATE_SPEECH: Category for hate speech. + *
+ * HARM_CATEGORY_SEXUALLY_EXPLICIT: Category for sexually explicit content. + *
+ * HARM_CATEGORY_HARASSMENT: Category for harassment. + *
+ * HARM_CATEGORY_DANGEROUS_CONTENT: Category for dangerous content. + * * @public */ export const HarmCategory = { @@ -40,40 +70,39 @@ export const HarmCategory = { /** * Harm categories that would cause prompts or candidates to be blocked. + * * @public */ export type HarmCategory = (typeof HarmCategory)[keyof typeof HarmCategory]; /** * Threshold above which a prompt or candidate will be blocked. + * + * @remarks + * BLOCK_LOW_AND_ABOVE: Content with `NEGLIGIBLE` will be allowed. + *
+ * BLOCK_MEDIUM_AND_ABOVE: Content with `NEGLIGIBLE` and `LOW` will be allowed. + *
+ * BLOCK_ONLY_HIGH: Content with `NEGLIGIBLE`, `LOW`, and `MEDIUM` will be allowed. + *
+ * BLOCK_NONE: All content will be allowed. + *
+ * OFF: All content will be allowed. This is the same as `BLOCK_NONE`, but the metadata corresponding + * to the `HarmCategory` will not be present in the response. + * * @public */ export const HarmBlockThreshold = { - /** - * Content with `NEGLIGIBLE` will be allowed. - */ BLOCK_LOW_AND_ABOVE: 'BLOCK_LOW_AND_ABOVE', - /** - * Content with `NEGLIGIBLE` and `LOW` will be allowed. - */ BLOCK_MEDIUM_AND_ABOVE: 'BLOCK_MEDIUM_AND_ABOVE', - /** - * Content with `NEGLIGIBLE`, `LOW`, and `MEDIUM` will be allowed. - */ BLOCK_ONLY_HIGH: 'BLOCK_ONLY_HIGH', - /** - * All content will be allowed. - */ BLOCK_NONE: 'BLOCK_NONE', - /** - * All content will be allowed. This is the same as `BLOCK_NONE`, but the metadata corresponding - * to the {@link (HarmCategory:type)} will not be present in the response. - */ OFF: 'OFF' } as const; /** * Threshold above which a prompt or candidate will be blocked. + * * @public */ export type HarmBlockThreshold = @@ -82,16 +111,15 @@ export type HarmBlockThreshold = /** * This property is not supported in the Gemini Developer API ({@link GoogleAIBackend}). * + * @remarks + * SEVERITY: The harm block method uses both probability and severity scores. + *
+ * PROBABILITY: The harm block method uses the probability score. + * * @public */ export const HarmBlockMethod = { - /** - * The harm block method uses both probability and severity scores. - */ SEVERITY: 'SEVERITY', - /** - * The harm block method uses the probability score. - */ PROBABILITY: 'PROBABILITY' } as const; @@ -105,29 +133,28 @@ export type HarmBlockMethod = /** * Probability that a prompt or candidate matches a harm category. + * + * @remarks + * NEGLIGIBLE: Content has a negligible chance of being unsafe. + *
+ * LOW: Content has a low chance of being unsafe. + *
+ * MEDIUM: Content has a medium chance of being unsafe. + *
+ * HIGH: Content has a high chance of being unsafe. + * * @public */ export const HarmProbability = { - /** - * Content has a negligible chance of being unsafe. - */ NEGLIGIBLE: 'NEGLIGIBLE', - /** - * Content has a low chance of being unsafe. - */ LOW: 'LOW', - /** - * Content has a medium chance of being unsafe. - */ MEDIUM: 'MEDIUM', - /** - * Content has a high chance of being unsafe. - */ HIGH: 'HIGH' } as const; /** * Probability that a prompt or candidate matches a harm category. + * * @public */ export type HarmProbability = @@ -135,138 +162,123 @@ export type HarmProbability = /** * Harm severity levels. + * + * @remarks + * HARM_SEVERITY_NEGLIGIBLE: Negligible level of harm severity. + *
+ * HARM_SEVERITY_LOW: Low level of harm severity. + *
+ * HARM_SEVERITY_MEDIUM: Medium level of harm severity. + *
+ * HARM_SEVERITY_HIGH: High level of harm severity. + *
+ * HARM_SEVERITY_UNSUPPORTED: Harm severity is not supported. The GoogleAI backend does not support `HarmSeverity`, so this value is used as a fallback. + * * @public */ export const HarmSeverity = { - /** - * Negligible level of harm severity. - */ HARM_SEVERITY_NEGLIGIBLE: 'HARM_SEVERITY_NEGLIGIBLE', - /** - * Low level of harm severity. - */ HARM_SEVERITY_LOW: 'HARM_SEVERITY_LOW', - /** - * Medium level of harm severity. - */ HARM_SEVERITY_MEDIUM: 'HARM_SEVERITY_MEDIUM', - /** - * High level of harm severity. - */ HARM_SEVERITY_HIGH: 'HARM_SEVERITY_HIGH', - /** - * Harm severity is not supported. - * - * @remarks - * The GoogleAI backend does not support `HarmSeverity`, so this value is used as a fallback. - */ HARM_SEVERITY_UNSUPPORTED: 'HARM_SEVERITY_UNSUPPORTED' } as const; /** * Harm severity levels. + * * @public */ export type HarmSeverity = (typeof HarmSeverity)[keyof typeof HarmSeverity]; /** * Reason that a prompt was blocked. + * + * @remarks + * SAFETY: Content was blocked by safety settings. + *
+ * OTHER: Content was blocked, but the reason is uncategorized. + *
+ * BLOCKLIST: Content was blocked because it contained terms from the terminology blocklist. + *
+ * PROHIBITED_CONTENT: Content was blocked due to prohibited content. + * * @public */ export const BlockReason = { - /** - * Content was blocked by safety settings. - */ SAFETY: 'SAFETY', - /** - * Content was blocked, but the reason is uncategorized. - */ OTHER: 'OTHER', - /** - * Content was blocked because it contained terms from the terminology blocklist. - */ BLOCKLIST: 'BLOCKLIST', - /** - * Content was blocked due to prohibited content. - */ PROHIBITED_CONTENT: 'PROHIBITED_CONTENT' } as const; /** * Reason that a prompt was blocked. + * * @public */ export type BlockReason = (typeof BlockReason)[keyof typeof BlockReason]; /** * Reason that a candidate finished. + * + * @remarks + * STOP: Natural stop point of the model or provided stop sequence. + *
+ * MAX_TOKENS: The maximum number of tokens as specified in the request was reached. + *
+ * SAFETY: The candidate content was flagged for safety reasons. + *
+ * RECITATION: The candidate content was flagged for recitation reasons. + *
+ * OTHER: Unknown reason. + *
+ * BLOCKLIST: The candidate content contained forbidden terms. + *
+ * PROHIBITED_CONTENT: The candidate content potentially contained prohibited content. + *
+ * SPII: The candidate content potentially contained Sensitive Personally Identifiable Information (SPII). + *
+ * MALFORMED_FUNCTION_CALL: The function call generated by the model was invalid. + * * @public */ export const FinishReason = { - /** - * Natural stop point of the model or provided stop sequence. - */ STOP: 'STOP', - /** - * The maximum number of tokens as specified in the request was reached. - */ MAX_TOKENS: 'MAX_TOKENS', - /** - * The candidate content was flagged for safety reasons. - */ SAFETY: 'SAFETY', - /** - * The candidate content was flagged for recitation reasons. - */ RECITATION: 'RECITATION', - /** - * Unknown reason. - */ OTHER: 'OTHER', - /** - * The candidate content contained forbidden terms. - */ BLOCKLIST: 'BLOCKLIST', - /** - * The candidate content potentially contained prohibited content. - */ PROHIBITED_CONTENT: 'PROHIBITED_CONTENT', - /** - * The candidate content potentially contained Sensitive Personally Identifiable Information (SPII). - */ SPII: 'SPII', - /** - * The function call generated by the model was invalid. - */ MALFORMED_FUNCTION_CALL: 'MALFORMED_FUNCTION_CALL' } as const; /** * Reason that a candidate finished. + * * @public */ export type FinishReason = (typeof FinishReason)[keyof typeof FinishReason]; /** + * @remarks + * - AUTO: Default model behavior; model decides to predict either a function call + * or a natural language response. + * - ANY: Model is constrained to always predicting a function call only. + * If `allowed_function_names` is set, the predicted function call will be + * limited to any one of `allowed_function_names`, else the predicted + * function call will be any one of the provided `function_declarations`. + *
+ * NONE: Model will not predict any function call. Model behavior is same as when + * not passing any function declarations. + * * @public */ export const FunctionCallingMode = { - /** - * Default model behavior; model decides to predict either a function call - * or a natural language response. - */ AUTO: 'AUTO', - /** - * Model is constrained to always predicting a function call only. - * If `allowed_function_names` is set, the predicted function call will be - * limited to any one of `allowed_function_names`, else the predicted - * function call will be any one of the provided `function_declarations`. - */ ANY: 'ANY', - /** - * Model will not predict any function call. Model behavior is same as when - * not passing any function declarations. - */ NONE: 'NONE' } as const; @@ -278,37 +290,34 @@ export type FunctionCallingMode = /** * Content part modality. + * + * @remarks + * MODALITY_UNSPECIFIED: Unspecified modality. + *
+ * TEXT: Plain text. + *
+ * IMAGE: Image. + *
+ * VIDEO: Video. + *
+ * AUDIO: Audio. + *
+ * DOCUMENT: Document (for example, PDF). + * * @public */ export const Modality = { - /** - * Unspecified modality. - */ MODALITY_UNSPECIFIED: 'MODALITY_UNSPECIFIED', - /** - * Plain text. - */ TEXT: 'TEXT', - /** - * Image. - */ IMAGE: 'IMAGE', - /** - * Video. - */ VIDEO: 'VIDEO', - /** - * Audio. - */ AUDIO: 'AUDIO', - /** - * Document (for example, PDF). - */ DOCUMENT: 'DOCUMENT' } as const; /** * Content part modality. + * * @public */ export type Modality = (typeof Modality)[keyof typeof Modality]; @@ -316,23 +325,18 @@ export type Modality = (typeof Modality)[keyof typeof Modality]; /** * Generation modalities to be returned in generation responses. * + * @remarks + * TEXT: Text. + *
+ * IMAGE: Image. + *
+ * AUDIO: Audio. + * * @beta */ export const ResponseModality = { - /** - * Text. - * @beta - */ TEXT: 'TEXT', - /** - * Image. - * @beta - */ IMAGE: 'IMAGE', - /** - * Audio. - * @beta - */ AUDIO: 'AUDIO' } as const; diff --git a/packages/ai/src/types/imagen/requests.ts b/packages/ai/src/types/imagen/requests.ts index 47d1afe3b01..691f1d137df 100644 --- a/packages/ai/src/types/imagen/requests.ts +++ b/packages/ai/src/types/imagen/requests.ts @@ -101,40 +101,37 @@ export interface ImagenGenerationConfig { /** * A filter level controlling how aggressively to filter sensitive content. * + * @remarks * Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI * are assessed against a list of safety filters, which include 'harmful categories' (for example, * `violence`, `sexual`, `derogatory`, and `toxic`). This filter level controls how aggressively to * filter out potentially harmful content from responses. See the {@link http://firebase.google.com/docs/vertex-ai/generate-images | documentation } * and the {@link https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#safety-filters | Responsible AI and usage guidelines} * for more details. + *
+ * BLOCK_LOW_AND_ABOVE: The most aggressive filtering level; most strict blocking. + *
+ * BLOCK_MEDIUM_AND_ABOVE: Blocks some sensitive prompts and responses. + *
+ * BLOCK_ONLY_HIGH: Blocks few sensitive prompts and responses. + *
+ * BLOCK_NONE: The least aggressive filtering level; blocks very few sensitive prompts and responses. + * Access to this feature is restricted and may require your case to be reviewed and approved by + * Cloud support. * * @beta */ export const ImagenSafetyFilterLevel = { - /** - * The most aggressive filtering level; most strict blocking. - */ BLOCK_LOW_AND_ABOVE: 'block_low_and_above', - /** - * Blocks some sensitive prompts and responses. - */ BLOCK_MEDIUM_AND_ABOVE: 'block_medium_and_above', - /** - * Blocks few sensitive prompts and responses. - */ BLOCK_ONLY_HIGH: 'block_only_high', - /** - * The least aggressive filtering level; blocks very few sensitive prompts and responses. - * - * Access to this feature is restricted and may require your case to be reviewed and approved by - * Cloud support. - */ BLOCK_NONE: 'block_none' } as const; /** * A filter level controlling how aggressively to filter sensitive content. * + * @remarks * Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI * are assessed against a list of safety filters, which include 'harmful categories' (for example, * `violence`, `sexual`, `derogatory`, and `toxic`). This filter level controls how aggressively to @@ -150,37 +147,34 @@ export type ImagenSafetyFilterLevel = /** * A filter level controlling whether generation of images containing people or faces is allowed. * + * @remarks * See the personGeneration * documentation for more details. + *
+ * BLOCK_ALL: Disallow generation of images containing people or faces; images of people are filtered out. + *
+ * ALLOW_ADULT: Allow generation of images containing adults only; images of children are filtered out. + * Generation of images containing people or faces may require your use case to be + * reviewed and approved by Cloud support; see the {@link https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#person-face-gen | Responsible AI and usage guidelines} + * for more details. + *
+ * ALLOW_ALL: Allow generation of images containing adults only; images of children are filtered out. + * Generation of images containing people or faces may require your use case to be + * reviewed and approved by Cloud support; see the {@link https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#person-face-gen | Responsible AI and usage guidelines} + * for more details. * * @beta */ export const ImagenPersonFilterLevel = { - /** - * Disallow generation of images containing people or faces; images of people are filtered out. - */ BLOCK_ALL: 'dont_allow', - /** - * Allow generation of images containing adults only; images of children are filtered out. - * - * Generation of images containing people or faces may require your use case to be - * reviewed and approved by Cloud support; see the {@link https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#person-face-gen | Responsible AI and usage guidelines} - * for more details. - */ ALLOW_ADULT: 'allow_adult', - /** - * Allow generation of images containing adults only; images of children are filtered out. - * - * Generation of images containing people or faces may require your use case to be - * reviewed and approved by Cloud support; see the {@link https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen#person-face-gen | Responsible AI and usage guidelines} - * for more details. - */ ALLOW_ALL: 'allow_all' } as const; /** * A filter level controlling whether generation of images containing people or faces is allowed. * + * @remarks * See the personGeneration * documentation for more details. * @@ -212,44 +206,37 @@ export interface ImagenSafetySettings { /** * Aspect ratios for Imagen images. * + * @remarks * To specify an aspect ratio for generated images, set the `aspectRatio` property in your - * {@link ImagenGenerationConfig}. - * - * See the {@link http://firebase.google.com/docs/vertex-ai/generate-images | documentation } + * {@link ImagenGenerationConfig}. See the {@link http://firebase.google.com/docs/vertex-ai/generate-images | documentation } * for more details and examples of the supported aspect ratios. + *
+ * SQUARE: Square (1:1) aspect ratio. + *
+ * LANDSCAPE_3x4: Landscape (3:4) aspect ratio. + *
+ * PORTRAIT_4x3: Portrait (4:3) aspect ratio. + *
+ * LANDSCAPE_16x9: Landscape (16:9) aspect ratio. + *
+ * PORTRAIT_9x16: Portrait (9:16) aspect ratio. * * @beta */ export const ImagenAspectRatio = { - /** - * Square (1:1) aspect ratio. - */ 'SQUARE': '1:1', - /** - * Landscape (3:4) aspect ratio. - */ 'LANDSCAPE_3x4': '3:4', - /** - * Portrait (4:3) aspect ratio. - */ 'PORTRAIT_4x3': '4:3', - /** - * Landscape (16:9) aspect ratio. - */ 'LANDSCAPE_16x9': '16:9', - /** - * Portrait (9:16) aspect ratio. - */ 'PORTRAIT_9x16': '9:16' } as const; /** * Aspect ratios for Imagen images. * + * @remarks * To specify an aspect ratio for generated images, set the `aspectRatio` property in your - * {@link ImagenGenerationConfig}. - * - * See the {@link http://firebase.google.com/docs/vertex-ai/generate-images | documentation } + * {@link ImagenGenerationConfig}. See the {@link http://firebase.google.com/docs/vertex-ai/generate-images | documentation } * for more details and examples of the supported aspect ratios. * * @beta diff --git a/packages/ai/src/types/responses.ts b/packages/ai/src/types/responses.ts index 8b8e1351675..a7a584ecc83 100644 --- a/packages/ai/src/types/responses.ts +++ b/packages/ai/src/types/responses.ts @@ -577,6 +577,13 @@ export interface LiveServerToolCallCancellation { /** * The types of responses that can be returned by {@link LiveSession.receive}. * + * @remarks + * SERVER_CONTENT: An incremental content update from the model. + *
+ * TOOL_CALL: A request from the model for the client to execute one or more functions. + *
+ * TOOL_CALL_CANCELLATION: Notification to cancel a previous function call triggered by {@link LiveServerToolCall}. + * * @beta */ export const LiveResponseType = { @@ -587,8 +594,6 @@ export const LiveResponseType = { /** * The types of responses that can be returned by {@link LiveSession.receive}. - * This is a property on all messages that can be used for type narrowing. This property is not - * returned by the server, it is assigned to a server message object once it's parsed. * * @beta */