Skip to content

Commit c74fb8e

Browse files
prachi-oktaprachi-okta
authored
Fix Flaky Integration Tests and CI Configuration Issues (#1655)
* fix: Correct page counting logic in testPagedIterableWithGroupMembers The test was failing because the page counting logic only incremented when collectedMembers.size() was divisible by the page limit (2). With an odd number of total members (e.g., 3), this resulted in pageCount staying at 1 instead of properly reflecting multiple pages. Fixed by: - Starting pageCount at 1 (since we always fetch at least one page) - Tracking previousSize to detect when we cross page boundaries - Incrementing pageCount when we've collected more items after hitting a page boundary (previousSize % 2 == 0) This ensures the test correctly validates that pagination is working across multiple API pages even when the total items is not evenly divisible by the page limit. * fix: Address PMD code quality violations - Collapsed nested if statements in MultiThreadingWarningUtil by combining conditions with && operator (lines 137-139 and 143-145) - Removed unnecessary local variable 'url' in PagedIterator.parseNextLinkFromHeaders, returning the value directly (line 129) These changes improve code readability and follow PMD best practices without changing functionality. * Adding delay for a testcases, and removing parallelism from CCI * Increase memory allocation for the ReversingLabs scan step * fix: Add proper eventual consistency handling for integration tests - GroupsIT: Add 3s wait after group creation for search indexing - UsersIT: Add 3s wait after user activation before listing - PaginationIT: Increase wait to 5s for user indexing in filter tests - IdpIT: Add comprehensive waits for IDP operations: * 3s after IDP creation * 2s after link/unlink operations * 2s before deletion - All delays use Math.max(getTestOperationDelay(), minimumMs) pattern Fixes flaky tests caused by eventual consistency in Okta API * fix: Use UUID for unique user emails and add delays to linkedInIdpTest - PaginationIT: Replace uniqueTestName with UUID for user emails to avoid conflicts - IdpIT: Add eventual consistency delays to linkedInIdpTest (same pattern as other IDP tests) This prevents 'user already exists' errors on test retries and fixes unlink timing issues * fix: Add more eventual consistency delays and retry logic - IdpIT.facebookIdpTest: Increase link wait from 2s to 5s (Facebook IDP slower) - UsersIT.createUserWithUserTypeTest: Add 2s delay after user creation - UsersIT.filterUserTest: Increase delay to 3s for search indexing - UsersIT.userSuspendTest: Add retry logic with exponential backoff for transient server errors (E0000009) These changes improve test reliability against eventual consistency and transient API errors * ci: Increase resources for ReversingLabs scan to prevent OOM - Add resource_class: xlarge to reversing-labs job (same as jdk11/jdk21) - Increase no_output_timeout to 45m for scanner step - Prevents SIGKILL (exit 9) when scanning 5,600+ files Fixes: ReversingLabs scanner consistently failing with memory exhaustion
1 parent 1daa922 commit c74fb8e

File tree

8 files changed

+131
-35
lines changed

8 files changed

+131
-35
lines changed

‎.circleci/config.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,17 +34,15 @@ jobs:
3434
- image: cimg/openjdk:11.0.22-node
3535
environment:
3636
JVM_OPTS: -Xmx3200m
37-
resource_class: large
38-
parallelism: 3
37+
resource_class: xlarge
3938
steps: *build_steps
4039

4140
jdk21:
4241
docker:
4342
- image: cimg/openjdk:21.0.2-node
4443
environment:
4544
JVM_OPTS: -Xmx3200m
46-
resource_class: large
47-
parallelism: 3
45+
resource_class: xlarge
4846
steps: *build_steps
4947

5048
snyk-scan:
@@ -63,6 +61,7 @@ jobs:
6361
reversing-labs:
6462
docker:
6563
- image: cimg/openjdk:21.0.2-node
64+
resource_class: xlarge
6665
steps:
6766
- checkout
6867
- run: npm install
@@ -97,6 +96,7 @@ jobs:
9796
# Run the wrapper, do not change anything here
9897
- run:
9998
name: Run Reversing Labs Wrapper Scanner
99+
no_output_timeout: 45m
100100
command: |
101101
echo "Scanning " ${CIRCLE_WORKING_DIRECTORY/#\~/$HOME}/api
102102
rl-wrapper \

‎api/src/main/java/com/okta/sdk/client/MultiThreadingWarningUtil.java‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -133,16 +133,14 @@ public void recordThreadAccess() {
133133
log.debug("New thread detected accessing Okta SDK: {} (ID: {}). Total unique threads: {}",
134134
currentThreadName, currentThreadId, threadCount);
135135

136-
if (threadCount > THREAD_COUNT_WARNING_THRESHOLD) {
137-
if (multiThreadWarningEmitted.compareAndSet(false, true)) {
138-
emitMultiThreadingWarning(threadCount);
139-
}
136+
if (threadCount > THREAD_COUNT_WARNING_THRESHOLD
137+
&& multiThreadWarningEmitted.compareAndSet(false, true)) {
138+
emitMultiThreadingWarning(threadCount);
140139
}
141140

142-
if (threadCount > 1 && isLikelyThreadPoolThread(currentThreadName)) {
143-
if (threadPoolWarningEmitted.compareAndSet(false, true)) {
144-
emitThreadPoolWarning();
145-
}
141+
if (threadCount > 1 && isLikelyThreadPoolThread(currentThreadName)
142+
&& threadPoolWarningEmitted.compareAndSet(false, true)) {
143+
emitThreadPoolWarning();
146144
}
147145
}
148146

‎api/src/main/java/com/okta/sdk/resource/client/PagedIterator.java‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -126,8 +126,7 @@ private String parseNextLinkFromHeaders(Map<String, List<String>> headers) {
126126
for (String linkHeader : linkHeaders) {
127127
Matcher matcher = NEXT_LINK_PATTERN.matcher(linkHeader);
128128
if (matcher.find()) {
129-
String url = matcher.group(1);
130-
return url; // The URL
129+
return matcher.group(1); // The URL
131130
}
132131
}
133132

‎integration-tests/src/test/groovy/com/okta/sdk/tests/it/GroupsIT.groovy‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,10 @@ class GroupsIT extends ITSupport {
7272
registerForCleanup(group)
7373
validateGroup(group, groupName)
7474

75-
// 2. Search the group by name
75+
// 2. Wait for eventual consistency - group needs to be indexed for search
76+
Thread.sleep(Math.max(getTestOperationDelay(), 3000))
77+
78+
// 3. Search the group by name
7679
assertGroupPresent(groupApi.listGroups(groupName, null, null, null, null, null, null, null), group)
7780
}
7881

@@ -156,7 +159,8 @@ class GroupsIT extends ITSupport {
156159

157160
// 3. Remove user from group and validate user removed
158161
groupApi.unassignUserFromGroup(group.getId(), user.getId())
159-
160-
assertUserNotInGroup(user, group, groupApi,10, getTestOperationDelay())
162+
// Wait for eventual consistency after removal
163+
sleep(3000)
164+
assertUserNotInGroup(user, group, groupApi, 20, getTestOperationDelay())
161165
}
162166
}

‎integration-tests/src/test/groovy/com/okta/sdk/tests/it/IdpIT.groovy‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -413,6 +413,9 @@ class IdpIT extends ITSupport {
413413
IdentityProvider createdIdp = identityProviderApi.createIdentityProvider(idp)
414414
registerForCleanup(createdIdp)
415415

416+
// Wait for eventual consistency - IDP needs time to be fully created
417+
Thread.sleep(Math.max(getTestOperationDelay(), 3000))
418+
416419
// list linked idp users
417420
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), hasSize(0))
418421

@@ -421,12 +424,18 @@ class IdpIT extends ITSupport {
421424
userIdentityProviderLinkRequest.setExternalId("external-id")
422425
identityProviderUsersApi.linkUserToIdentityProvider(createdIdp.getId(), createdUser.getId(), userIdentityProviderLinkRequest)
423426

427+
// Wait for link operation to complete
428+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
429+
424430
// list linked idp users
425431
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), hasSize(1))
426432

427433
// unlink user
428434
identityProviderUsersApi.unlinkUserFromIdentityProvider(createdIdp.getId(), createdUser.getId())
429435

436+
// Wait for unlink operation to complete
437+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
438+
430439
// list linked idp users
431440
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), hasSize(0))
432441

@@ -437,6 +446,9 @@ class IdpIT extends ITSupport {
437446
// deactivate
438447
identityProviderApi.deactivateIdentityProvider(createdIdp.getId())
439448

449+
// Wait before delete
450+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
451+
440452
// delete
441453
identityProviderApi.deleteIdentityProvider(createdIdp.getId())
442454
}
@@ -500,6 +512,9 @@ class IdpIT extends ITSupport {
500512
IdentityProvider createdIdp = identityProviderApi.createIdentityProvider(idp)
501513
registerForCleanup(createdIdp)
502514

515+
// Wait for eventual consistency - IDP needs time to be fully created
516+
Thread.sleep(Math.max(getTestOperationDelay(), 3000))
517+
503518
// list linked idp users
504519
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
505520

@@ -508,12 +523,18 @@ class IdpIT extends ITSupport {
508523
userIdentityProviderLinkRequest.setExternalId("external-id")
509524
identityProviderUsersApi.linkUserToIdentityProvider(createdIdp.getId(), createdUser.getId(), userIdentityProviderLinkRequest)
510525

526+
// Wait for link operation to complete
527+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
528+
511529
// list linked idp users
512530
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(1))
513531

514532
// unlink user
515533
identityProviderUsersApi.unlinkUserFromIdentityProvider(createdIdp.getId(), createdUser.getId())
516534

535+
// Wait for unlink operation to complete
536+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
537+
517538
// list linked idp users
518539
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
519540

@@ -583,6 +604,9 @@ class IdpIT extends ITSupport {
583604
IdentityProvider createdIdp = identityProviderApi.createIdentityProvider(idp)
584605
registerForCleanup(createdIdp)
585606

607+
// Wait for eventual consistency - IDP needs time to be fully created
608+
Thread.sleep(Math.max(getTestOperationDelay(), 3000))
609+
586610
// list linked idp users
587611
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
588612

@@ -591,18 +615,27 @@ class IdpIT extends ITSupport {
591615
userIdentityProviderLinkRequest.setExternalId("external-id")
592616
identityProviderUsersApi.linkUserToIdentityProvider(createdIdp.getId(), createdUser.getId(), userIdentityProviderLinkRequest)
593617

618+
// Wait for link operation to complete - Facebook IDP may need more time
619+
Thread.sleep(Math.max(getTestOperationDelay(), 5000))
620+
594621
// list linked idp users
595622
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(1))
596623

597624
// unlink user
598625
identityProviderUsersApi.unlinkUserFromIdentityProvider(createdIdp.getId(), createdUser.getId())
599626

627+
// Wait for unlink operation to complete
628+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
629+
600630
// list linked idp users
601631
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
602632

603633
// deactivate
604634
identityProviderApi.deactivateIdentityProvider(createdIdp.getId())
605635

636+
// Wait before delete
637+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
638+
606639
// delete
607640
identityProviderApi.deleteIdentityProvider(createdIdp.getId())
608641
}
@@ -666,6 +699,9 @@ class IdpIT extends ITSupport {
666699
IdentityProvider createdIdp = identityProviderApi.createIdentityProvider(idp)
667700
registerForCleanup(createdIdp)
668701

702+
// Wait for eventual consistency - IDP needs time to be fully created
703+
Thread.sleep(Math.max(getTestOperationDelay(), 3000))
704+
669705
// list linked idp users
670706
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
671707

@@ -674,18 +710,27 @@ class IdpIT extends ITSupport {
674710
userIdentityProviderLinkRequest.setExternalId("external-id")
675711
identityProviderUsersApi.linkUserToIdentityProvider(createdIdp.getId(), createdUser.getId(), userIdentityProviderLinkRequest)
676712

713+
// Wait for link operation to complete
714+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
715+
677716
// list linked idp users
678717
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(1))
679718

680719
// unlink user
681720
identityProviderUsersApi.unlinkUserFromIdentityProvider(createdIdp.getId(), createdUser.getId())
682721

722+
// Wait for unlink operation to complete
723+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
724+
683725
// list linked idp users
684726
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
685727

686728
// deactivate
687729
identityProviderApi.deactivateIdentityProvider(createdIdp.getId())
688730

731+
// Wait before delete
732+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
733+
689734
// delete
690735
identityProviderApi.deleteIdentityProvider(createdIdp.getId())
691736
}
@@ -749,6 +794,9 @@ class IdpIT extends ITSupport {
749794
IdentityProvider createdIdp = identityProviderApi.createIdentityProvider(idp)
750795
registerForCleanup(createdIdp)
751796

797+
// Wait for eventual consistency - IDP needs time to be fully created
798+
Thread.sleep(Math.max(getTestOperationDelay(), 3000))
799+
752800
// list linked idp users
753801
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
754802

@@ -757,18 +805,27 @@ class IdpIT extends ITSupport {
757805
userIdentityProviderLinkRequest.setExternalId("external-id")
758806
identityProviderUsersApi.linkUserToIdentityProvider(createdIdp.getId(), createdUser.getId(), userIdentityProviderLinkRequest)
759807

808+
// Wait for link operation to complete
809+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
810+
760811
// list linked idp users
761812
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(1))
762813

763814
// unlink user
764815
identityProviderUsersApi.unlinkUserFromIdentityProvider(createdIdp.getId(), createdUser.getId())
765816

817+
// Wait for unlink operation to complete
818+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
819+
766820
// list linked idp users
767821
assertThat(identityProviderUsersApi.listIdentityProviderApplicationUsers(createdIdp.getId(), null, null, null, null), iterableWithSize(0))
768822

769823
// deactivate
770824
identityProviderApi.deactivateIdentityProvider(createdIdp.getId())
771825

826+
// Wait before delete
827+
Thread.sleep(Math.max(getTestOperationDelay(), 2000))
828+
772829
// delete
773830
identityProviderApi.deleteIdentityProvider(createdIdp.getId())
774831
}

‎integration-tests/src/test/groovy/com/okta/sdk/tests/it/PaginationIT.groovy‎

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -204,8 +204,9 @@ class PaginationIT extends ITSupport {
204204

205205
try {
206206
println "Creating ${usersToAdd} users and adding to group..."
207+
def uniqueSuffix = UUID.randomUUID().toString().take(8)
207208
for (int i = 0; i < usersToAdd; i++) {
208-
def email = "pagmem${i}-${uniqueTestName.take(30)}@ex.com"
209+
def email = "pagmem${i}-${uniqueSuffix}@example.com"
209210
User user = createUser(userApi, email, "MemberTest${i}", "User${i}")
210211
createdUsers.add(user)
211212
registerForCleanup(user)
@@ -214,19 +215,34 @@ class PaginationIT extends ITSupport {
214215
groupApi.assignUserToGroup(createdGroup.id, user.id)
215216
}
216217

217-
Thread.sleep(getTestOperationDelay())
218+
// Wait for eventual consistency - users need time to be added to group
219+
println "Waiting for users to be added to group (eventual consistency)..."
220+
Thread.sleep(Math.max(getTestOperationDelay(), 5000))
221+
222+
// Verify at least some users were added before proceeding
223+
def initialCheck = groupApi.listGroupUsers(createdGroup.id, null, null).toList()
224+
println "Initial check found ${initialCheck.size()} members in group"
225+
if (initialCheck.size() == 0) {
226+
println "No members found, waiting additional 5 seconds..."
227+
Thread.sleep(5000)
228+
}
218229

219230
println "Fetching group members with PagedIterable (limit=2 per page)..."
220231
def collectedMembers = []
221-
def pageCount = 0
232+
def pageCount = 1 // Start at 1 since we'll fetch at least one page
233+
def previousSize = 0
222234

223235
// Use listGroupUsersPaged with limit=2
224236
for (User member : groupApi.listGroupUsersPaged(createdGroup.id, null, 2)) {
225237
collectedMembers.add(member)
226-
if (collectedMembers.size() % 2 == 0) {
238+
// Increment page count when we've fetched a new batch (size increases by more than 0 after hitting limit boundary)
239+
if (previousSize > 0 && previousSize % 2 == 0 && collectedMembers.size() > previousSize) {
227240
pageCount++
228241
println " Fetched page ${pageCount} (${collectedMembers.size()} total members so far)"
242+
} else if (collectedMembers.size() % 2 == 0) {
243+
println " Fetched page ${pageCount} (${collectedMembers.size()} total members so far)"
229244
}
245+
previousSize = collectedMembers.size()
230246
}
231247

232248
println "✓ Collected ${collectedMembers.size()} members across ${pageCount} pages"
@@ -315,13 +331,14 @@ class PaginationIT extends ITSupport {
315331
println "\n=== Testing Filtered Pagination ==="
316332
UserApi userApi = new UserApi(getClient())
317333

318-
def email = "pagfilt-${uniqueTestName.take(30)}@ex.com"
334+
def email = "pagfilt-${UUID.randomUUID().toString().take(8)}@example.com"
319335
println "Creating user: ${email}"
320336
User createdUser = createUser(userApi, email, "FilterTest", "User")
321337
registerForCleanup(createdUser)
322338

323-
// Allow time for indexing
324-
Thread.sleep(getTestOperationDelay())
339+
// Wait for eventual consistency - user needs to be indexed for search/filter
340+
println "Waiting for user to be indexed (eventual consistency)..."
341+
Thread.sleep(Math.max(getTestOperationDelay(), 5000))
325342

326343
println "Fetching users with filter: profile.email eq \"${email}\""
327344
// Use filter with PagedIterable

0 commit comments

Comments
 (0)