You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -243,26 +228,26 @@ An Azure Batch job is a logical group of similar tasks. In our scenario, it is "
243
228
These tasks would run in parallel and deployed across multiple nodes, orchestrated by the Azure Batch service.
244
229
245
230
> [!TIP]
246
-
> You can use the taskSlotsPerNode property to specify maximum number of tasks that can run concurrently on a single node.
231
+
> You can use the [taskSlotsPerNode](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/batch/arm-batch/src/models/index.ts#L1190-L1191) property to specify maximum number of tasks that can run concurrently on a single node.
247
232
248
233
#### Preparation task
249
234
250
235
The VM nodes created are blank Ubuntu nodes. Often, you need to install a set of programs as prerequisites.
251
236
Typically, for Linux nodes you can have a shell script that installs the prerequisites before the actual tasks run. However it could be any programmable executable.
252
237
253
-
The [shell script](https://github.com/shwetams/azure-batchclient-sample-nodejs/blob/master/startup_prereq.sh) in this example installs Python-pip and the Azure Storage SDK for Python.
238
+
The [shell script](https://github.com/Azure-Samples/azure-batch-samples/blob/master/JavaScript/Node.js/startup_prereq.sh) in this example installs Python-pip and the Azure Storage Blob SDK for Python.
254
239
255
240
You can upload the script on an Azure Storage Account and generate a SAS URI to access the script. This process can also be automated using the Azure Storage JavaScript SDK.
256
241
257
242
> [!TIP]
258
-
> A preparation task for a job runs only on the VM nodes where the specific task needs to run. If you want prerequisites to be installed on all nodes irrespective of the tasks that run on it, you can use the startTask property while adding a pool. You can use the following preparation task definition for reference.
243
+
> A preparation task for a job runs only on the VM nodes where the specific task needs to run. If you want prerequisites to be installed on all nodes irrespective of the tasks that run on it, you can use the [startTask](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/batch/batch/src/models/index.ts#L1432) property while adding a pool. You can use the following preparation task definition for reference.
259
244
260
-
A preparation task is specified during the submission of Azure Batch job. Following are the preparation task configuration parameters:
245
+
A preparation task is specified during the submission of Azure Batch job. Following are some configurable preparation task parameters:
261
246
262
247
-**ID**: A unique identifier for the preparation task
263
248
-**commandLine**: Command line to execute the task executable
264
249
-**resourceFiles**: Array of objects that provide details of files needed to be downloaded for this task to run. Following are its options
265
-
-blobSource: The SAS URI of the file
250
+
-httpUrl: The URL of the file to download
266
251
- filePath: Local path to download and save the file
267
252
- fileMode: Only applicable for Linux nodes, fileMode is in octal format with a default value of 0770
268
253
-**waitForSuccess**: If set to true, the task does not run on preparation task failures
@@ -271,64 +256,71 @@ A preparation task is specified during the submission of Azure Batch job. Follow
271
256
Following code snippet shows the preparation task script configuration sample:
272
257
273
258
```javascript
274
-
varjob_prep_task_config= {id:"installprereq",commandLine:"sudo sh startup_prereq.sh > startup.log",resourceFiles:[{'blobSource':'Blob SAS URI','filePath':'startup_prereq.sh'}],waitForSuccess:true,runElevated:true}
259
+
varjobPrepTaskConfig= {id:"installprereq",commandLine:"sudo sh startup_prereq.sh > startup.log",resourceFiles: [{ 'httpUrl':'Blob sh url', 'filePath':'startup_prereq.sh'}],waitForSuccess:true,runElevated:true, userIdentity: {autoUser: {elevationLevel:"admin", scope:"pool"}}}
275
260
```
276
261
277
262
If there are no prerequisites to be installed for your tasks to run, you can skip the preparation tasks. Following code creates a job with display name "process csv files."
278
263
279
264
```javascript
280
-
// Setting up Batch pool configuration
281
-
var pool_config = {poolId:poolid}
282
-
// Setting up Job configuration along with preparation task
283
-
var jobId ="processcsvjob"
284
-
var job_config = {id:jobId,displayName:"process csv files",jobPreparationTask:job_prep_task_config,poolInfo:pool_config}
265
+
// Setting Batch Pool ID
266
+
constpoolInfo= { poolId: poolId };
267
+
// Batch job configuration object
268
+
constjobId="processcsvjob";
269
+
constjobConfig= {
270
+
id: jobId,
271
+
displayName:"process csv files",
272
+
jobPreparationTask: jobPrepTaskConfig,
273
+
poolInfo: poolInfo
274
+
};
285
275
// Adding Azure batch job to the pool
286
-
var job =batch_client.job.add(job_config,function(error,result){
constjob=batchClient.job.add(jobConfig, function (error, result) {
277
+
if (error !==null) {
278
+
console.log("An error occurred while creating the job...");
279
+
console.log(error.response);
280
+
}
281
+
});
291
282
```
292
283
293
284
### Step 5: Submit Azure Batch tasks for a job
294
285
295
286
Now that our process csv job is created, let us create tasks for that job. Assuming we have four containers, we have to create four tasks, one for each container.
296
287
297
-
If we look at the [Python script](https://github.com/shwetams/azure-batchclient-sample-nodejs/blob/master/processcsv.py), it accepts two parameters:
288
+
If we look at the [Python script](https://github.com/Azure-Samples/azure-batch-samples/blob/master/JavaScript/Node.js/processcsv.py), it accepts two parameters:
298
289
299
290
- container name: The Storage container to download files from
300
291
- pattern: An optional parameter of file name pattern
301
292
302
-
Assuming we have four containers "con1", "con2", "con3","con4" following code shows submitting for tasks to the Azure batch job "process csv" we created earlier.
293
+
Assuming we have four containers "con1", "con2", "con3","con4" following code shows submitting four tasks to the Azure batch job "process csv" we created earlier.
303
294
304
295
```javascript
305
296
// storing container names in an array
306
-
var container_list= ["con1","con2","con3","con4"]
307
-
container_list.forEach(function(val,index){
308
-
309
-
var container_name= val;
310
-
vartaskID =container_name+"_process";
311
-
var task_config = {id:taskID,displayName:'process csv in '+ container_name,commandLine:'python processcsv.py --container '+ container_name,resourceFiles:[{'blobSource':'<blob SAS URI>','filePath':'processcsv.py'}]}
312
-
var task =batch_client.task.add(poolid,task_config,function(error,result){
313
-
if(error !=null)
314
-
{
315
-
console.log(error.response);
316
-
}
317
-
else
318
-
{
319
-
console.log("Task for container : "+ container_name +"submitted successfully");
320
-
}
321
-
322
-
323
-
324
-
});
325
-
297
+
constcontainerList= ["con1","con2","con3","con4"]; //Replace with list of blob containers within storage account
298
+
containerList.forEach(function(val,index){
299
+
console.log("Submitting task for container : "+ val);
consttask=batchClient.task.add(jobId, taskConfig, function (error, result) {
311
+
if (error !==null) {
312
+
console.log("Error occured while creating task for container "+ containerName +". Details : "+error.response);
313
+
}
314
+
else {
315
+
console.log("Task for container : "+ containerName +" submitted successfully");
316
+
}
326
317
});
318
+
});
327
319
```
328
320
329
321
The code adds multiple tasks to the pool. And each of the tasks is executed on a node in the pool of VMs created. If the number of tasks exceeds the number of VMs in a pool or the taskSlotsPerNode property, the tasks wait until a node is made available. This orchestration is handled by Azure Batch automatically.
330
322
331
-
The portal has detailed views on the tasks and job statuses. You can also use the list and get functions in the Azure JavaScript SDK..
323
+
The portal has detailed views on the tasks and job statuses. You can also use the list and get functions in the Azure JavaScript SDK. Details are provided in the documentation [link](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/batch/batch/src/operations/job.ts#L114-L149).
0 commit comments