Skip to content

Commit be33610

Browse files
chore(glue): timeout and worker type validation for Ray jobs (aws#32119)
### Issue # (if applicable) Closes aws#29612. ### Reason for this change AWS Glue Ray job has some restriction. - must use Z.2X worker type ```sh CREATE_FAILED [...] Worker type cannot be null and only [Z.2X] worker types are supported for glueray jobs ``` - must not specify timeout ```sh UPDATE_FAILED [...] Timeout not supported for Ray jobs ``` ### Description of changes Add validation for above restriction. ```ts if (executable.type.name === JobType.RAY.name) { if (props.workerType !== WorkerType.Z_2X) { throw new Error(`WorkerType must be Z_2X for Ray jobs, got: ${props.workerType}`); } if (props.timeout !== undefined) { throw new Error('Timeout cannot be set for Ray jobs'); } } ``` ### Description of how you validated changes Add unit test. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent c768554 commit be33610

File tree

2 files changed

+43
-0
lines changed

2 files changed

+43
-0
lines changed

packages/@aws-cdk/aws-glue-alpha/lib/job.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -738,6 +738,17 @@ export class Job extends JobBase {
738738
}
739739
}
740740

741+
// Validate Ray job properties
742+
// See https://github.com/aws/aws-cdk/issues/29612
743+
if (executable.type.name === JobType.RAY.name) {
744+
if (props.workerType !== WorkerType.Z_2X) {
745+
throw new Error(`WorkerType must be Z_2X for Ray jobs, got: ${props.workerType}`);
746+
}
747+
if (props.timeout !== undefined) {
748+
throw new Error('Timeout cannot be set for Ray jobs');
749+
}
750+
}
751+
741752
let maxCapacity = props.maxCapacity;
742753
if (maxCapacity !== undefined && (props.workerType && props.workerCount !== undefined)) {
743754
throw new Error('maxCapacity cannot be used when setting workerType and workerCount');

packages/@aws-cdk/aws-glue-alpha/test/job.test.ts

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -896,6 +896,38 @@ describe('Job', () => {
896896
workerCount: 2,
897897
})).toThrow('Runtime is required for Ray jobs');
898898
});
899+
900+
test.each([
901+
glue.WorkerType.G_025X,
902+
glue.WorkerType.G_1X,
903+
glue.WorkerType.G_2X,
904+
glue.WorkerType.G_4X,
905+
glue.WorkerType.G_8X,
906+
])('throw error for unsupported worker type', (workerType) => {
907+
expect(() => new glue.Job(stack, 'Job', {
908+
executable: glue.JobExecutable.pythonRay({
909+
glueVersion: glue.GlueVersion.V4_0,
910+
pythonVersion: glue.PythonVersion.THREE_NINE,
911+
runtime: glue.Runtime.RAY_TWO_FOUR,
912+
script,
913+
}),
914+
workerType,
915+
workerCount: 2,
916+
})).toThrow(`WorkerType must be Z_2X for Ray jobs, got: ${workerType}`);
917+
});
918+
});
919+
920+
test('throw error for specifying timeout', () => {
921+
expect(() => new glue.Job(stack, 'Job', {
922+
executable: glue.JobExecutable.pythonRay({
923+
glueVersion: glue.GlueVersion.V4_0,
924+
pythonVersion: glue.PythonVersion.THREE_NINE,
925+
runtime: glue.Runtime.RAY_TWO_FOUR,
926+
script,
927+
}),
928+
workerType: glue.WorkerType.Z_2X,
929+
timeout: cdk.Duration.minutes(5),
930+
})).toThrow('Timeout cannot be set for Ray jobs');
899931
});
900932

901933
test('etl job with all props should synthesize correctly', () => {

0 commit comments

Comments
 (0)