feat(specs): add external injection source definition and runtime parameter #5283

ben-kalmus · 2025-08-28T22:27:35Z

🧭 What and Why

🎟 JIRA Ticket:

Changes included:

Add external injection search source
injectedItems can now either be a search or external source.
Add injectedItems compositions runtime parameter.

🧪 Test

CI

algolia-bot · 2025-08-28T22:27:49Z

✔️ Code generated!

Name	Link
🪓 Triggered by	`07c0b9dbc6e8c6e79f4859dbd414c7c8c8d566fe`
🍃 Generated commit	`90129e77ffb45ccdbf3dd6449c97e25c95fc568d`
🌲 Generated branch	`generated/benk/compositions/external-source`

📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language	Req/s
go	1752
php	1514
javascript	1445
csharp	1282
python	1143
java	1005
ruby	815
swift	400
scala	23

ClaraMuller

Small comment but first shot is super clean 👍

specs/composition-full/common/params/Composition.yml

specs/composition-full/common/schemas/components/InjectionSource.yml

specs/composition-full/common/params/Composition.yml

gavinwade12 · 2025-08-29T13:19:54Z

specs/composition-full/common/params/Composition.yml

+    A list of injected objectIDs into an external source.
+  default: []
+  example:
+    [{'objectID': 'my-object-1', 'metadata': {'my-field': 'my-value'}}]


Have you verified the generated examples look correct? Typically JSON requires " instead of '

Not sure how to generate examples

we don't generate examples or tests from the specs, any test created in the CTS is then exposed a fully validated code snippet that can be exposed on the docs

you can find the output here #5283 (comment) when the CI is done running, under the docs/snippets folder

gavinwade12 · 2025-08-29T13:24:12Z

specs/composition-full/common/schemas/components/CompositionBehavior.yml

-      required:
-        - search
+      description: Search source to be used to inject items into result set.
+      $ref: '#/injectionSource'


nit: the previous name made it clear the type was only good for injectedItems. I think you could even keep it inline here since it can't be reused outside injectedItems

Suggested change

$ref: '#/injectionSource'

title: injectedItemSource

oneOf:

- $ref: './InjectionSource.yml#/search'

- $ref: './InjectionSource.yml#/external'

Changed the name, but had to keep 'oneOf' statement as root component. It cannot be nested, as I'm getting an error:
56:5 error oneOf must be out of line automation-custom/out-of-line-one-of

gavinwade12 · 2025-08-29T13:24:48Z

specs/composition-full/common/schemas/components/InjectionSource.yml

+    - type: object
+      additionalProperties: false


Do you need this object w/ no properties?

Dropped this

gavinwade12 · 2025-08-29T16:52:15Z

specs/composition-full/common/params/Composition.yml

+    objectID:
+      type: string
+    metadata:
+      $ref: '../schemas/components/CompositionBehavior.yml#/metadata'


This points to the entire metadata object, but it should really just be generic key-value pairs. I think technically something like this might work, but there's probably a better way

$ref: '../schemas/components/CompositionBehavior.yml#/metadata/properties/hits/properties/extra'

Removed reference

specs/composition-full/common/schemas/components/InjectionSource.yml

specs/composition-full/common/params/Composition.yml

ClaraMuller

Small comment but LGTM otherwise

specs/composition-full/common/params/Composition.yml

millotp · 2025-09-02T08:08:54Z

Hello, there is an issue with the CTS, I think you need to update the putComposition.json file:

Error in putComposition: Parameter 'search' not found in 'source'. Available properties are: index, params.

ben-kalmus · 2025-09-02T09:00:01Z

Hello, there is an issue with the CTS, I think you need to update the putComposition.json file:
Error in putComposition: Parameter 'search' not found in 'source'. Available properties are: index, params.

I've fixed this, the error was a little deceptive because the issue was with injectedItemSource in CompositionBehavior.json. However, when I generate typescript locally, I don't quite see the result that I should expect. For example, in external.ts

import type { Search } from './search';

/**
 * Injected items will originate from externally provided objectIDs (that must exist in the index) given at runtime in the run request payload.
 */
export type External = Search;

The External type is just an alias for Search, however the External source should also have an additional field ordering | string.

millotp · 2025-09-02T09:11:58Z

Unfortunately the generator does not support anonymous objects, because it's not possible to generate in all the languages that we support.
In this allOf, the ordering property should be defined in it's own object, with it's own title.
Then you can use it in the allOf like:

allOf:
  - $ref: '#/search'
  - $ref: '#/ordering'

ben-kalmus · 2025-09-02T09:18:51Z

Unfortunately the generator does not support anonymous objects, because it's not possible to generate in all the languages that we support. In this allOf, the ordering property should be defined in it's own object, with it's own title. Then you can use it in the allOf like:
allOf:
  - $ref: '#/search'
  - $ref: '#/ordering'

That worked! Thank you.

Co-authored-by: Clara Muller <[email protected]>

millotp

thanks for the contribution !

ben-kalmus force-pushed the benk/compositions/external-source branch 2 times, most recently from 3c21967 to 61a020b Compare August 29, 2025 09:04

ben-kalmus marked this pull request as ready for review August 29, 2025 10:00

ben-kalmus requested a review from a team as a code owner August 29, 2025 10:00

ben-kalmus requested review from a team, millotp and shortcuts August 29, 2025 10:00

ClaraMuller requested changes Aug 29, 2025

View reviewed changes

ClaraMuller requested a review from a team August 29, 2025 10:25

ben-kalmus force-pushed the benk/compositions/external-source branch from d443f32 to d58c58f Compare August 29, 2025 15:58

gavinwade12 reviewed Aug 29, 2025

View reviewed changes

specs/composition-full/common/schemas/components/InjectionSource.yml Outdated Show resolved Hide resolved

ClaraMuller reviewed Sep 1, 2025

View reviewed changes

specs/composition-full/common/params/Composition.yml Outdated Show resolved Hide resolved

ben-kalmus force-pushed the benk/compositions/external-source branch from 866333f to 5c5302b Compare September 1, 2025 12:07

ClaraMuller previously approved these changes Sep 1, 2025

View reviewed changes

specs/composition-full/common/params/Composition.yml Outdated Show resolved Hide resolved

ben-kalmus dismissed ClaraMuller’s stale review via eb81d39 September 1, 2025 13:54

ben-kalmus force-pushed the benk/compositions/external-source branch 3 times, most recently from dc532ae to d1d482f Compare September 1, 2025 16:58

ben-kalmus force-pushed the benk/compositions/external-source branch 2 times, most recently from 94b9ff7 to e0441d1 Compare September 2, 2025 13:11

ben-kalmus added 3 commits September 2, 2025 16:13

add external source definition and runtime param

53582f4

fix injection description

82f84e5

reuse metadata from source

58ec1ab

ben-kalmus and others added 8 commits September 2, 2025 16:13

code review fixes

14b2b6a

Update external injected item specs for metadata

3fa2032

Co-authored-by: Clara Muller <[email protected]>

fix injectedItemSource

8e0566a

fix external missing ordering field

bec4b2e

add test examples

4a2c596

fix run param injectedItems

78ec01e

add search|external layer

983e482

rename wrapper

9e65302

ben-kalmus force-pushed the benk/compositions/external-source branch from 483c3dc to 9e65302 Compare September 2, 2025 15:13

millotp approved these changes Sep 2, 2025

View reviewed changes

millotp merged commit 646b7a5 into main Sep 2, 2025
29 checks passed

millotp deleted the benk/compositions/external-source branch September 2, 2025 15:38

-      $ref: '#/injectionSource'
+      title: injectedItemSource
+      oneOf:
+            - $ref: './InjectionSource.yml#/search'
+            - $ref: './InjectionSource.yml#/external'

feat(specs): add external injection source definition and runtime parameter #5283

feat(specs): add external injection source definition and runtime parameter #5283

Uh oh!

Conversation

ben-kalmus commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧭 What and Why

Changes included:

🧪 Test

Uh oh!

algolia-bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✔️ Code generated!

Uh oh!

ClaraMuller left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gavinwade12 Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ClaraMuller left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

millotp commented Sep 2, 2025

Uh oh!

ben-kalmus commented Sep 2, 2025

Uh oh!

millotp commented Sep 2, 2025

Uh oh!

ben-kalmus commented Sep 2, 2025

Uh oh!

millotp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

ben-kalmus commented Aug 28, 2025 •

edited

Loading

algolia-bot commented Aug 28, 2025 •

edited

Loading

gavinwade12 Aug 29, 2025 •

edited

Loading