@@ -105,27 +105,54 @@ This auto-indexing-specific library defines the following two functions.
105105 - Type:
106106 ```
107107 ({
108+ -- List of patterns to match against paths in the repository
108109 "patterns": array[pattern],
110+ -- List of patterns to match against paths in the repository
111+ -- for getting contents (see contents_by_path below)
109112 "patterns_for_content": array[pattern],
110- "generate": (registration_api, paths: array[string], contents_by_path: table[string, string]) -> array[index_job],
113+ -- Callback function invoked with paths requested by patterns above
114+ -- for creating index jobs
115+ "generate": (
116+ registration_api,
117+ -- List of paths obtained from 'patterns' and
118+ -- 'patterns_for_content' combined.
119+ paths: array[string],
120+ -- Table mapping paths to contents for paths matched by
121+ -- 'patterns_for_content'
122+ contents_by_path: table[string, string]
123+ ) -> array[index_job],
111124 }) -> recognizer
112125 ```
113126 where ` index_job ` is an object with the following shape:
114127 ```
115128 index_job = {
116- "indexer": string, -- Docker image for the indexer
117- "root": string, -- working directory for invoking the indexer
118- "steps": array[{ -- preparatory steps to run before invoking the indexer (e.g. installing dependencies)
119- "root": string, -- working directory for this step
120- "image": string -- Docker image to use for preparatory step
121- "commands": array[string] -- List of commands to run inside the Docker image
129+ -- Docker image for the indexer
130+ "indexer": string,
131+ -- Working directory for invoking the indexer
132+ "root": string,
133+ -- Preparatory steps to run before invoking the indexer
134+ -- such as installing dependencies
135+ "steps": array[{
136+ -- Working directory for this step
137+ "root": string,
138+ -- Docker image to use for this step
139+ "image": string,
140+ -- List of commands to run inside the Docker image
141+ "commands": array[string]
122142 }],
123- "local_steps": array[string] -- List of commands to run inside the indexer image at "root" before invoking
124- -- the indexer (e.g. to install dependencies)
125- "indexer_args": array[string], -- command-line invocation for the indexer
126- "outfile": string, -- path to the index generated by the indexer
127- "requested_envvars": array[string], -- List of environment variables needed. These are made accessible
128- -- to steps, local_steps, and the indexer_args command.
143+ -- List of commands to run inside the indexer image at "root"
144+ -- before invoking the indexer, such as installing dependencies.
145+ "local_steps": array[string],
146+ -- Command-line invocation for the indexer
147+ "indexer_args": array[string],
148+ -- Path to the index generated by the indexer
149+ "outfile": string,
150+ -- Names of necessary environment variables. These are
151+ -- made accessible to steps, local_steps, and the
152+ -- indexer_args command.
153+ --
154+ -- These are generally used for passing secrets.
155+ "requested_envvars": array[string],
129156 }
130157 ```
131158 For installing dependencies, if the indexer image contains the relevant package manager(s),
0 commit comments