feat: Support AI Extract endpoints by Hawra2020 · Pull Request #574 · box/boxcli

Hawra2020 · 2025-04-28T13:43:30Z

No description provided.

CLAassistant · 2025-04-28T13:43:38Z

All committers have signed the CLA.

coveralls · 2025-04-28T13:49:50Z

Pull Request Test Coverage Report for Build 14999587544

Details

117 of 135 (86.67%) changed or added relevant lines in 3 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.1%) to 85.38%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/commands/ai/extract.js	31	33	93.94%
src/box-command.js	20	27	74.07%
src/commands/ai/extract-structured.js	66	75	88.0%

Files with Coverage Reduction	New Missed Lines	%
src/box-command.js	1	75.72%

Totals
Change from base Build 14168264651:	0.1%
Covered Lines:	4507
Relevant Lines:	5091

💛 - Coveralls

docs/ai.md

src/commands/ai/extract-structured.js

congminh1254 · 2025-05-08T09:08:14Z

src/commands/ai/extract.js

+  'Sends an AI request to supported LLMs and extracts metadata in the form of key value pairs';
+AiExtractCommand.examples = [
+  'box ai:extract --items=id=12345,type=file --prompt "firstName, lastName, location, yearOfBirth, company"',
+  'box ai:extract --items=id=12345,type=file --prompt "firstName, lastName" --ai_agent="id=14031;type=ai_agent_extract;basic_text.llm_endpoint_params.type=openai_params;basic_text.llm_endpoint_params.frequency_penalty=1.5;basic_text.llm_endpoint_params.presence_penalty=1.5;basic_text.llm_endpoint_params.stop=<|im_end|>;basic_text.llm_endpoint_params.temperature=0;basic_text.llm_endpoint_params.top_p=1;basic_text.model=azure__openai__gpt_4o_mini;basic_text.num_tokens_for_completion=8400;basic_text.prompt_template=It is {current_date}, consider these travel options {content} and answer the {user_question}.;basic_text.system_message=You are a helpful travel assistant specialized in budget travel;long_text.embeddings.model=azure__openai__text_embedding_ada_002;long_text.embeddings.strategy.id=basic;long_text.embeddings.strategy.num_tokens_per_chunk=64;long_text.llm_endpoint_params.type=openai_params;long_text.llm_endpoint_params.frequency_penalty=1.5;long_text.llm_endpoint_params.presence_penalty=1.5;long_text.llm_endpoint_params.stop=<|im_end|>;long_text.llm_endpoint_params.temperature=0;long_text.llm_endpoint_params.top_p=1;long_text.model=azure__openai__gpt_4o_mini;long_text.num_tokens_for_completion=8400;long_text.prompt_template=It is {current_date}, consider these travel options {content} and answer the {user_question}.;long_text.system_message=You are a helpful travel assistant specialized in budget travel"',


I think the AI Agent should be a JSON object?

congminh1254 · 2025-05-08T09:08:38Z

src/commands/ai/extract.js

+  })
+};
+
+module.exports = AiExtractCommand;


Alway keep newline at end of file

congminh1254 · 2025-05-08T09:09:40Z

test/commands/ai.test.js

+			});
+	});
+
+	describe('ai:extract-structured', () => {


You also need add another test with AI Agent including in the params

congminh1254 · 2025-05-08T09:35:53Z

test/commands/ai.test.js

 const assert = require('chai').assert;
-const { TEST_API_ROOT, getFixture } = require('../helpers/test-helper');
+const {TEST_API_ROOT, getFixture} = require('../helpers/test-helper');
+const fs = require('fs');


congminh1254 · 2025-05-09T07:34:07Z

src/box-command.js

+ */
+
+function removeUndefinedValues(obj) {


We have warning in Github Action Run:

congminh1254 · 2025-05-09T07:35:32Z

docs/ai.md

+FLAGS
+  -h, --help                       Show CLI help
+  -q, --quiet                      Suppress any non-error output to stderr
+  -s, --save                       Save report to default reports folder on disk
+  -t, --token=<value>              Provide a token to perform this call
+  -v, --verbose                    Show verbose output, which can be helpful for debugging
+  -y, --yes                        Automatically respond yes to all confirmation prompts
+      --as-user=<value>            Provide an ID for a user
+      --bulk-file-path=<value>     File path to bulk .csv or .json objects
+      --csv                        Output formatted CSV
+      --fields=<value>             Comma separated list of fields to show
+      --items=<value>...           (required) The items for the AI request
+      --json                       Output formatted JSON
+      --no-color                   Turn off colors for logging
+      --prompt=<value>             (required) The prompt for the AI request
+      --save-to-file-path=<value>  Override default file path to save report


Why we don't have the AI Agent in this list of flag?

congminh1254 · 2025-05-09T07:36:59Z

docs/ai.md

+
+```
+USAGE
+  $ box ai:extract --prompt <value> --items <value>... --ai_agent <value> [-t <value>] [--as-user <value>] [--no-color] [--json |


The flag ai_agent is inconsistent with other params, normally we use kebab case for flags, like no-color or as-user,...

congminh1254

Please keep this coding practice:

Always newline at end of file.
For comment, please have a spacing and uppercase for first character, like

// This is comment

but not

//this is comment

congminh1254 · 2025-05-09T07:39:45Z

Please check the failing CI before asking for review PR

congminh1254

I pushed some change about eslint, please pull before fixing your code.

src/commands/ai/extract-structured.js

congminh1254 · 2025-05-12T09:16:50Z

src/commands/ai/extract-structured.js

+	}
+}
+
+AiExtractStructuredCommand.description = 'Extract structured metadata from a file using Box AI';


The description should reuse from Box API, like: "Sends an AI request to supported Large Language Models (LLMs) and returns extracted metadata as a set of key-value pairs."

congminh1254 · 2025-05-12T09:17:52Z

src/commands/ai/extract-structured.js

+	'box ai:extract-structured --items="id=12345,type=file" --fields "key=firstName,type=string,description=Person first name,prompt=What is the first name?,displayName=First name" --fields "key=lastName,type=string,description=Person last name,prompt=What is the last name?,displayName=Last name" --ai-agent \'{"type":"ai_agent_extract","basicText":{"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<|im_end|>","temperature": 0,"topP": 1},"model": "azure__openai__gpt_4o_mini","numTokensForCompletion": 8400,"promptTemplate": "It is, consider these travel options and answer the.","systemMessage": "You are a helpful travel assistant specialized in budget travel"},"longText":{"embeddings":{ "model": "azure__openai__text_embedding_ada_002","strategy":{"id": "basic","numTokensPerChunk": 64}},"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<|im_end|>","temperature": 0,"topP": 1},"model":"azure__openai__gpt_4o_mini","numTokensForCompletion":8400,"promptTemplate":"It is , consider these travel options and answer the.","systemMessage":"You are a helpful travel assistant specialized in budget travel"}}\'',
+	'box ai:extract-structured --items="id=12345,type=file" --metadata-template="type=metadata_template,scope=enterprise,template_key=test" --ai-agent \'{"type":"ai_agent_extract","basicText":{"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<|im_end|>","temperature": 0,"topP": 1},"model": "azure__openai__gpt_4o_mini","numTokensForCompletion": 8400,"promptTemplate": "It is, consider these travel options and answer the.","systemMessage": "You are a helpful travel assistant specialized in budget travel"},"longText":{"embeddings":{ "model": "azure__openai__text_embedding_ada_002","strategy":{"id": "basic","numTokensPerChunk": 64}},"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<|im_end|>","temperature": 0,"topP": 1},"model":"azure__openai__gpt_4o_mini","numTokensForCompletion":8400,"promptTemplate":"It is , consider these travel options and answer the.","systemMessage":"You are a helpful travel assistant specialized in budget travel"}}\'',


The example is too complicated and hard to read, should only one example (without the AI Agent) or the second example can with the AI Agent but with a few main fields.

congminh1254 · 2025-05-12T09:20:18Z

src/commands/ai/extract-structured.js

+		},
+	}),
+	'metadata-template': Flags.string({
+		description: 'metadata template to use for the AI request',


Upper case for Metadata.

For most of the description, you should reuse the description we have from the Box API:

And you can let user know which field is able to put into this metadata template field.

congminh1254 · 2025-05-12T09:20:35Z

src/commands/ai/extract-structured.js

+	fields: Flags.string({
+		multiple: true,
+		description:
+			'JSON string of fields to extract (e.g., [{"key":"firstName","type":"string","description":"Person first name","prompt":"What is the first name?","displayName":"First name"}])',


Not a JSON string

congminh1254 · 2025-05-12T09:21:52Z

src/commands/ai/extract-structured.js

+			const fields = {
+				key: '',
+				type: '',
+				description: '',
+				prompt: '',
+				displayName: '',
+			};
+
+			const obj = utils.parseStringToObject(input, ['key', 'type', 'description', 'prompt', 'displayName']);
+			for (const key in obj) {
+				if (key === 'key') {
+					fields.key = obj[key];
+				} else if (key === 'type') {
+					fields.type = obj[key];
+				} else if (key === 'description') {
+					fields.description = obj[key];
+				} else if (key === 'prompt') {
+					fields.prompt = obj[key];
+				} else if (key === 'displayName') {
+					fields.displayName = obj[key];
+				} else {
+					throw new Error(`Invalid item key ${key}`);
+				}
+			}


Missing options.

congminh1254 · 2025-05-12T09:22:18Z

src/commands/ai/extract-structured.js

+	}),
+	'ai-agent': Flags.string({
+		required: false,
+		description: 'The AI agent to be used for extraction (e.g., key=value pairs with semicolons or file:config.json)',


Not in key=value format

congminh1254 · 2025-05-12T09:23:03Z

src/commands/ai/extract.js

+	...BoxCommand.flags,
+	prompt: Flags.string({
+		required: true,
+		description: 'The prompt for the AI request',


Reuse description from Box API, but short version (should the first sentence, for example)

congminh1254 · 2025-05-12T09:23:06Z

src/commands/ai/extract.js

+	}),
+	items: Flags.string({
+		required: true,
+		description: 'The items for the AI request',


Reuse description from Box API, but short version (should the first sentence, for example)

congminh1254 · 2025-05-12T09:23:42Z

src/commands/ai/extract.js

+	}),
+	'ai-agent': Flags.string({
+		required: false,
+		description: 'The AI agent to be used for extraction',


Should let user know it's in JSON format and can provide an simple example for this field

congminh1254 · 2025-05-13T09:14:38Z

src/commands/ai/extract-structured.js

+AiExtractStructuredCommand.description = 'Sends an AI request to supported Large Language Models (LLMs) and returns extracted metadata as a set of key-value pairs. For this request, you either need a metadata template or a list of fields you want to extract. Input is either a metadata template or a list of fields to ensure the structure.';
+AiExtractStructuredCommand.examples = [
+	    'box ai:extract-structured --items="id=12345,type=file" --fields "key=hobby,type=multiSelect,description=Person hobby,prompt=What is your hobby?,displayName=Hobby,options=Guitar;Books"',
+		'box ai:extract-structured --items="id=12345,type=file" --fields "key=firstName,type=string,description=Person first name,prompt=What is the first name?,displayName=First name" --fields "key=lastName,type=string,description=Person last name,prompt=What is the last name?,displayName=Last name"',


Remove this example

congminh1254 · 2025-05-13T09:15:34Z

src/commands/ai/extract-structured.js

+AiExtractStructuredCommand.examples = [
+	    'box ai:extract-structured --items="id=12345,type=file" --fields "key=hobby,type=multiSelect,description=Person hobby,prompt=What is your hobby?,displayName=Hobby,options=Guitar;Books"',
+		'box ai:extract-structured --items="id=12345,type=file" --fields "key=firstName,type=string,description=Person first name,prompt=What is the first name?,displayName=First name" --fields "key=lastName,type=string,description=Person last name,prompt=What is the last name?,displayName=Last name"',
+		'box ai:extract-structured --items="id=12345,type=file" --metadata-template="type=metadata_template,scope=enterprise,template_key=test" --ai-agent \'{"type":"ai_agent_extract_structured","basicText":{"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<|im_end|>","temperature": 0,"topP": 1},"model": "azure__openai__gpt_4o_mini","numTokensForCompletion": 8400,"promptTemplate": "It is, consider these travel options and answer the.","systemMessage": "You are a helpful travel assistant specialized in budget travel"},"longText":{"embeddings":{ "model": "azure__openai__text_embedding_ada_002","strategy":{"id": "basic","numTokensPerChunk": 64}},"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<|im_end|>","temperature": 0,"topP": 1},"model":"azure__openai__gpt_4o_mini","numTokensForCompletion":8400,"promptTemplate":"It is , consider these travel options and answer the.","systemMessage":"You are a helpful travel assistant specialized in budget travel"}}\'',


Remove long text

congminh1254 · 2025-05-13T09:17:35Z

src/commands/ai/extract-structured.js

+		multiple: true,
+		description: 'The fields to be extracted from the provided items.',
+		parse(input) {
+			const fields = {


Don't assign empty string by default

congminh1254 · 2025-05-13T09:19:40Z

src/commands/ai/extract.js

+		required: false,
+		description: 'The AI agent to be used for the extraction.',
+		parse(input) {
+			return JSON.parse(input);


Add try catch

congminh1254 · 2025-05-13T09:20:00Z

src/commands/ai/extract-structured.js

+			try {
+				return JSON.parse(input);
+			} catch (error) {
+				throw ('Error parsing ai agent ', error);


AI in upper case

congminh1254 · 2025-05-13T09:20:28Z

src/commands/ai/extract.js

+	}),
+	'ai-agent': Flags.string({
+		required: false,
+		description: 'The AI agent to be used for the extraction.',


Should be JSON, and example

congminh1254 · 2025-05-13T09:22:37Z

test/commands/ai.test.js

+					description: 'Person first name',
+					prompt: 'What is the first name?',
+					displayName: 'First name',
+					options: [],


Add testing for the options

congminh1254 reviewed Apr 28, 2025

View reviewed changes

docs/ai.md Outdated Show resolved Hide resolved

congminh1254 reviewed May 8, 2025

View reviewed changes

src/commands/ai/extract-structured.js Show resolved Hide resolved

congminh1254 reviewed May 8, 2025

View reviewed changes

congminh1254 reviewed May 9, 2025

View reviewed changes

src/box-command.js

*/

function removeUndefinedValues(obj) {

Copy link

Member

congminh1254 May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have warning in Github Action Run:

congminh1254 reviewed May 9, 2025

View reviewed changes

Hawra2020 changed the title ~~feat: Introduce new endpoints to cli sdk for /ai/extract and /ai/extract structured~~ feat: Introduce new endpoints to CLI SDK for /ai/extract and /ai/extract structured May 9, 2025

congminh1254 reviewed May 12, 2025

View reviewed changes

congminh1254 reviewed May 13, 2025

View reviewed changes

Hawra2020 force-pushed the sdk-3983-introduce-new-endpoints-to-cli-sdk-for-/ai/extract-and-/ai/extract_structured branch from 3e3365a to 3082107 Compare May 13, 2025 10:00

feat: Support AI Extract endpoints

71a29c4

Hawra2020 force-pushed the sdk-3983-introduce-new-endpoints-to-cli-sdk-for-/ai/extract-and-/ai/extract_structured branch from 3082107 to 71a29c4 Compare May 13, 2025 13:17

congminh1254 force-pushed the sdk-3983-introduce-new-endpoints-to-cli-sdk-for-/ai/extract-and-/ai/extract_structured branch from 16935da to 71a29c4 Compare May 13, 2025 14:01

congminh1254 added 2 commits May 13, 2025 16:03

Update docs

a45cc7b

Update markdown

5cf2710

congminh1254 previously approved these changes May 13, 2025

View reviewed changes

congminh1254 dismissed their stale review via 75c906f May 13, 2025 14:39

congminh1254 force-pushed the sdk-3983-introduce-new-endpoints-to-cli-sdk-for-/ai/extract-and-/ai/extract_structured branch from 75c906f to 5cf2710 Compare May 13, 2025 14:42

congminh1254 closed this May 13, 2025

congminh1254 reopened this May 13, 2025

congminh1254 changed the title ~~feat: Introduce new endpoints to CLI SDK for /ai/extract and /ai/extract structured~~ feat: Support AI Extract endpoints May 13, 2025

congminh1254 approved these changes May 13, 2025

View reviewed changes

congminh1254 merged commit 0b4ff6b into main May 13, 2025
27 checks passed

congminh1254 deleted the sdk-3983-introduce-new-endpoints-to-cli-sdk-for-/ai/extract-and-/ai/extract_structured branch May 13, 2025 15:11

		'box ai:extract-structured --items="id=12345,type=file" --fields "key=firstName,type=string,description=Person first name,prompt=What is the first name?,displayName=First name" --fields "key=lastName,type=string,description=Person last name,prompt=What is the last name?,displayName=Last name" --ai-agent \'{"type":"ai_agent_extract","basicText":{"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<\|im_end\|>","temperature": 0,"topP": 1},"model": "azure__openai__gpt_4o_mini","numTokensForCompletion": 8400,"promptTemplate": "It is, consider these travel options and answer the.","systemMessage": "You are a helpful travel assistant specialized in budget travel"},"longText":{"embeddings":{ "model": "azure__openai__text_embedding_ada_002","strategy":{"id": "basic","numTokensPerChunk": 64}},"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<\|im_end\|>","temperature": 0,"topP": 1},"model":"azure__openai__gpt_4o_mini","numTokensForCompletion":8400,"promptTemplate":"It is , consider these travel options and answer the.","systemMessage":"You are a helpful travel assistant specialized in budget travel"}}\'',
		'box ai:extract-structured --items="id=12345,type=file" --metadata-template="type=metadata_template,scope=enterprise,template_key=test" --ai-agent \'{"type":"ai_agent_extract","basicText":{"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<\|im_end\|>","temperature": 0,"topP": 1},"model": "azure__openai__gpt_4o_mini","numTokensForCompletion": 8400,"promptTemplate": "It is, consider these travel options and answer the.","systemMessage": "You are a helpful travel assistant specialized in budget travel"},"longText":{"embeddings":{ "model": "azure__openai__text_embedding_ada_002","strategy":{"id": "basic","numTokensPerChunk": 64}},"llmEndpointParams":{"type":"openai_params","frequencyPenalty": 1.5,"presencePenalty": 1.5,"stop": "<\|im_end\|>","temperature": 0,"topP": 1},"model":"azure__openai__gpt_4o_mini","numTokensForCompletion":8400,"promptTemplate":"It is , consider these travel options and answer the.","systemMessage":"You are a helpful travel assistant specialized in budget travel"}}\'',

Conversation

Hawra2020 commented Apr 28, 2025

Uh oh!

CLAassistant commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 14999587544

Details

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

congminh1254 left a comment

Choose a reason for hiding this comment

Uh oh!

congminh1254 commented May 9, 2025

Uh oh!

congminh1254 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

CLAassistant commented Apr 28, 2025 •

edited

Loading

coveralls commented Apr 28, 2025 •

edited

Loading