-
-
Notifications
You must be signed in to change notification settings - Fork 24.6k
feat: add MuAPI image generation tool node #6527
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Anil-matcha
wants to merge
4
commits into
FlowiseAI:main
Choose a base branch
from
Anil-matcha:feat/muapi-image-tool
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+192
−0
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| import { INodeParams, INodeCredential } from '../src/Interface' | ||
|
|
||
| class MuApiApi implements INodeCredential { | ||
| label: string | ||
| name: string | ||
| version: number | ||
| inputs: INodeParams[] | ||
|
|
||
| constructor() { | ||
| this.label = 'MuAPI API' | ||
| this.name = 'muApiApi' | ||
| this.version = 1.0 | ||
| this.inputs = [ | ||
| { | ||
| label: 'MuAPI API Key', | ||
| name: 'muApiKey', | ||
| type: 'password', | ||
| description: 'Get your API key at https://muapi.ai/dashboard/api-keys' | ||
| } | ||
| ] | ||
| } | ||
| } | ||
|
|
||
| module.exports = { credClass: MuApiApi } |
164 changes: 164 additions & 0 deletions
164
packages/components/nodes/tools/MuAPIImage/MuAPIImage.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,164 @@ | ||||||||||||||||
| import { DynamicStructuredTool } from '@langchain/core/tools' | ||||||||||||||||
| import { z } from 'zod' | ||||||||||||||||
| import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface' | ||||||||||||||||
| import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils' | ||||||||||||||||
|
|
||||||||||||||||
| const BASE_URL = 'https://api.muapi.ai/api/v1' | ||||||||||||||||
|
|
||||||||||||||||
| const IMAGE_MODELS = [ | ||||||||||||||||
| 'flux-schnell', | ||||||||||||||||
| 'flux-dev', | ||||||||||||||||
| 'flux-kontext-dev', | ||||||||||||||||
| 'flux-kontext-pro', | ||||||||||||||||
| 'flux-kontext-max', | ||||||||||||||||
| 'hidream-fast', | ||||||||||||||||
| 'hidream-dev', | ||||||||||||||||
| 'hidream-full', | ||||||||||||||||
| 'midjourney', | ||||||||||||||||
| 'gpt4o', | ||||||||||||||||
| 'gpt-image-2', | ||||||||||||||||
| 'imagen4', | ||||||||||||||||
| 'imagen4-fast', | ||||||||||||||||
| 'seedream', | ||||||||||||||||
| 'reve', | ||||||||||||||||
| 'ideogram', | ||||||||||||||||
| 'hunyuan', | ||||||||||||||||
| 'wan2.1', | ||||||||||||||||
| 'qwen' | ||||||||||||||||
| ] as const | ||||||||||||||||
|
|
||||||||||||||||
| async function submitAndPoll(apiKey: string, endpoint: string, payload: object): Promise<string> { | ||||||||||||||||
| if (!apiKey) { | ||||||||||||||||
| throw new Error('MuAPI API key is not configured. Please set it in the credential.') | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| const submitResp = await fetch(`${BASE_URL}/${endpoint}`, { | ||||||||||||||||
| method: 'POST', | ||||||||||||||||
| headers: { 'x-api-key': apiKey, 'Content-Type': 'application/json' }, | ||||||||||||||||
| body: JSON.stringify(payload) | ||||||||||||||||
| }) | ||||||||||||||||
| if (!submitResp.ok) { | ||||||||||||||||
| const err = await submitResp.text() | ||||||||||||||||
| throw new Error(`MuAPI submit error (${submitResp.status}): ${err}`) | ||||||||||||||||
| } | ||||||||||||||||
| const submitData = await submitResp.json() | ||||||||||||||||
| const request_id: string | undefined = submitData?.request_id | ||||||||||||||||
| if (!request_id) { | ||||||||||||||||
| throw new Error(`MuAPI did not return a request_id. Response: ${JSON.stringify(submitData)}`) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| const deadline = Date.now() + 300_000 | ||||||||||||||||
| while (Date.now() < deadline) { | ||||||||||||||||
| await new Promise((r) => setTimeout(r, 3000)) | ||||||||||||||||
| const pollResp = await fetch(`${BASE_URL}/predictions/${request_id}/result`, { | ||||||||||||||||
| headers: { 'x-api-key': apiKey } | ||||||||||||||||
| }) | ||||||||||||||||
| if (!pollResp.ok) throw new Error(`MuAPI poll error (${pollResp.status})`) | ||||||||||||||||
| const data = await pollResp.json() | ||||||||||||||||
| if (!data) throw new Error('MuAPI poll returned an empty response') | ||||||||||||||||
| if (data.status === 'completed') { | ||||||||||||||||
| const outputs: string[] = data.outputs ?? [] | ||||||||||||||||
| if (!outputs.length) throw new Error('Generation completed but returned no outputs') | ||||||||||||||||
| return outputs[0] | ||||||||||||||||
| } | ||||||||||||||||
| if (data.status === 'failed' || data.status === 'cancelled') { | ||||||||||||||||
| throw new Error(`Generation ${data.status}: ${data.error ?? ''}`) | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
| throw new Error('MuAPI generation timed out after 5 minutes') | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| class MuAPIImage_Tools implements INode { | ||||||||||||||||
| label: string | ||||||||||||||||
| name: string | ||||||||||||||||
| version: number | ||||||||||||||||
| description: string | ||||||||||||||||
| type: string | ||||||||||||||||
| icon: string | ||||||||||||||||
| category: string | ||||||||||||||||
| baseClasses: string[] | ||||||||||||||||
| credential: INodeParams | ||||||||||||||||
| inputs: INodeParams[] | ||||||||||||||||
|
|
||||||||||||||||
| constructor() { | ||||||||||||||||
| this.label = 'MuAPI Image Generation' | ||||||||||||||||
| this.name = 'muAPIImage' | ||||||||||||||||
| this.version = 1.0 | ||||||||||||||||
| this.type = 'MuAPIImage' | ||||||||||||||||
| this.icon = 'muapi.svg' | ||||||||||||||||
| this.category = 'Tools' | ||||||||||||||||
| this.description = | ||||||||||||||||
| 'Generate images from text prompts using muapi.ai — a unified API for 400+ models including Flux, Midjourney, GPT-4o Image, Imagen 4, Seedream, HiDream, and more.' | ||||||||||||||||
| this.credential = { | ||||||||||||||||
| label: 'Connect Credential', | ||||||||||||||||
| name: 'credential', | ||||||||||||||||
| type: 'credential', | ||||||||||||||||
| credentialNames: ['muApiApi'] | ||||||||||||||||
| } | ||||||||||||||||
| this.inputs = [ | ||||||||||||||||
| { | ||||||||||||||||
| label: 'Model', | ||||||||||||||||
| name: 'model', | ||||||||||||||||
| type: 'options', | ||||||||||||||||
| options: IMAGE_MODELS.map((m) => ({ label: m, name: m })), | ||||||||||||||||
| default: 'flux-schnell', | ||||||||||||||||
| description: 'The image generation model to use' | ||||||||||||||||
| }, | ||||||||||||||||
| { | ||||||||||||||||
| label: 'Tool Name', | ||||||||||||||||
| name: 'toolName', | ||||||||||||||||
| type: 'string', | ||||||||||||||||
| default: 'generate_image', | ||||||||||||||||
| description: 'Name of the tool as exposed to the LLM agent' | ||||||||||||||||
| }, | ||||||||||||||||
| { | ||||||||||||||||
| label: 'Tool Description', | ||||||||||||||||
| name: 'toolDescription', | ||||||||||||||||
| type: 'string', | ||||||||||||||||
| rows: 3, | ||||||||||||||||
| default: | ||||||||||||||||
| 'Generate an image from a text description using muapi.ai. Returns the URL of the generated image.', | ||||||||||||||||
| description: 'Description shown to the LLM to guide tool use' | ||||||||||||||||
| } | ||||||||||||||||
| ] | ||||||||||||||||
| this.baseClasses = [this.type, ...getBaseClasses(DynamicStructuredTool)] | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> { | ||||||||||||||||
| const credentialData = await getCredentialData(nodeData.credential ?? '', options) | ||||||||||||||||
| const apiKey = getCredentialParam('muApiKey', credentialData, nodeData) | ||||||||||||||||
|
Comment on lines
+128
to
+129
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The
Suggested change
References
|
||||||||||||||||
|
|
||||||||||||||||
| if (!apiKey) { | ||||||||||||||||
| throw new Error('MuAPI API key is missing. Please configure it in the credential settings.') | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| const model = (nodeData.inputs?.model as string) ?? 'flux-schnell' | ||||||||||||||||
| const toolName = (nodeData.inputs?.toolName as string) ?? 'generate_image' | ||||||||||||||||
| const toolDescription = | ||||||||||||||||
| (nodeData.inputs?.toolDescription as string) ?? | ||||||||||||||||
| 'Generate an image from a text description using muapi.ai. Returns the URL of the generated image.' | ||||||||||||||||
|
|
||||||||||||||||
| return new DynamicStructuredTool({ | ||||||||||||||||
| name: toolName, | ||||||||||||||||
| description: toolDescription, | ||||||||||||||||
| schema: z.object({ | ||||||||||||||||
| prompt: z.string().describe('Text description of the image to generate'), | ||||||||||||||||
| width: z.number().optional().describe('Image width in pixels'), | ||||||||||||||||
| height: z.number().optional().describe('Image height in pixels') | ||||||||||||||||
| }), | ||||||||||||||||
| func: async ({ prompt, width, height }) => { | ||||||||||||||||
| const payload: Record<string, unknown> = { prompt } | ||||||||||||||||
| if (width) payload.width = width | ||||||||||||||||
| if (height) payload.height = height | ||||||||||||||||
| try { | ||||||||||||||||
| const url = await submitAndPoll(apiKey, model, payload) | ||||||||||||||||
| return JSON.stringify({ image_url: url, model, prompt }) | ||||||||||||||||
| } catch (e: any) { | ||||||||||||||||
| return `Error: ${e.message}` | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
| }) | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| module.exports = { nodeClass: MuAPIImage_Tools } | ||||||||||||||||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the poll response body is empty or null, accessing
data.statuswill throw aTypeError. We should add a nullish check fordatabefore accessing its properties, adhering to the repository's loose equality (== null) standard for nullish checks.References
== null) as a standard idiom for a 'nullish' check that covers bothnullandundefined.