SNOW-1926861: Run lob tests with 128 MB limit #831

sfc-gh-dprzybysz · 2025-02-13T07:59:18Z

teamwork issue 1230
To fix the failures, retry on most curl errors except particular ones (same as ODBC).

sfc-gh-ext-simba-hx · 2025-12-08T17:32:24Z

Hi @sfc-gh-pbulawa
Will look into the test failure. Since extending timeout didn't work I'll turn on debug logging to collect more information of the failure and would need your help to collect the output on Jenkins.

sfc-gh-ext-simba-hx · 2025-12-17T17:25:12Z

Hi @sfc-gh-pbulawa
Could you help to check what's the failure on Jenkins?

sfc-gh-pbulawa · 2025-12-18T07:47:07Z

@sfc-gh-ext-simba-hx,
aarch64 on gcp:

**** 44
      Start 44: ****_lob

44: Test command: /mnt/host/cmake-build-Release/****s/****_lob
44: Working Directory: /mnt/host/cmake-build-Release/****s
44: Test timeout computed to be: 10000000
44: [==========] Running 10 ****(s).
44: [ RUN      ] ****_lob_retrieval_arrow
44: [       OK ] ****_lob_retrieval_arrow
44: [ RUN      ] ****_lob_literal_arrow
44: [       OK ] ****_lob_literal_arrow
44: [ RUN      ] ****_lob_positional_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, /mnt/host/lib/http_perform.c, Line, 586
44: [       OK ] ****_lob_positional_bind_arrow
44: [ RUN      ] ****_lob_named_bind_arrow
44: [       OK ] ****_lob_named_bind_arrow
44: [ RUN      ] ****_lob_describe_only_arrow
44: [       OK ] ****_lob_describe_only_arrow
44: [ RUN      ] ****_lob_retrieval_json
44: [       OK ] ****_lob_retrieval_json
44: [ RUN      ] ****_lob_literal_json
44: [       OK ] ****_lob_literal_json
44: [ RUN      ] ****_lob_positional_bind_json
44: [       OK ] ****_lob_positional_bind_json
44: [ RUN      ] ****_lob_named_bind_json
44: [       OK ] ****_lob_named_bind_json
44: [ RUN      ] ****_lob_describe_only_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, /mnt/host/lib/http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a98b != 0
44: [   LINE   ] --- /mnt/host/****s/****_lob.c:356: error: Failure!
44: [  FAILED  ] ****_lob_describe_only_json
44: [==========] 10 ****(s) run.
44: [  PASSED  ] 9 ****(s).
44: [  FAILED  ] 1 ****(s), listed below:
44: [  FAILED  ] ****_lob_describe_only_json
44: 
44:  1 FAILED TEST(S)
44/87 Test #44: ****_lob ..............................***Failed  1821.66 sec

win32 on gcp:

**** 44
      Start 44: ****_lob

44: Test command: C:\jenkins\workspace\LibSfClient-Win32-VS17-Release-Test\cmake-build-win32-VS17-Release\****s\Release\****_lob.exe
44: Working Directory: C:/jenkins/workspace/LibSfClient-Win32-VS17-Release-Test/cmake-build-win32-VS17-Release/****s
44: Test timeout computed to be: 10000000
44: [==========] Running 10 ****(s).
44: [ RUN      ] ****_lob_retrieval_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_retrieval_arrow
44: [ RUN      ] ****_lob_literal_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_literal_arrow
44: [ RUN      ] ****_lob_positional_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_positional_bind_arrow
44: [ RUN      ] ****_lob_named_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_named_bind_arrow
44: [ RUN      ] ****_lob_describe_only_arrow
44: [       OK ] ****_lob_describe_only_arrow
44: [ RUN      ] ****_lob_retrieval_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_retrieval_json
44: [ RUN      ] ****_lob_literal_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win32-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_literal_json
44: [ RUN      ] ****_lob_positional_bind_json
Build timed out (after 120 minutes). Marking the build as failed.
Build was aborted

win64 on gcp:

**** 44
      Start 44: ****_lob

44: Test command: C:\jenkins\workspace\LibSfClient-Win64-VS17-Release-Test\cmake-build-win64-VS17-Release\****s\Release\****_lob.exe
44: Working Directory: C:/jenkins/workspace/LibSfClient-Win64-VS17-Release-Test/cmake-build-win64-VS17-Release/****s
44: Test timeout computed to be: 10000000
44: [==========] Running 10 ****(s).
44: [ RUN      ] ****_lob_retrieval_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_retrieval_arrow
44: [ RUN      ] ****_lob_literal_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_literal_arrow
44: [ RUN      ] ****_lob_positional_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_positional_bind_arrow
44: [ RUN      ] ****_lob_named_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_named_bind_arrow
44: [ RUN      ] ****_lob_describe_only_arrow
44: [       OK ] ****_lob_describe_only_arrow
44: [ RUN      ] ****_lob_retrieval_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_retrieval_json
44: [ RUN      ] ****_lob_literal_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [  ERROR   ] --- 0x3a980 != 0x0
44: [   LINE   ] --- C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\****s\****_lob.c:76: error: Failure!
44: [  FAILED  ] ****_lob_literal_json
44: [ RUN      ] ****_lob_positional_bind_json
Build timed out (after 120 minutes). Marking the build as failed.
Build was aborted

sfc-gh-ext-simba-hx · 2025-12-22T18:19:25Z

Hi @sfc-gh-pbulawa
Sorry for keep bothering you on this but please help to collect the failure information again.
I think I've made fixes for the failures you mentioned previously but the test on Jenkins still failing.

sfc-gh-pbulawa · 2025-12-23T11:44:32Z

Hi @sfc-gh-ext-simba-hx, here are the logs. This time only x64 failed, I've rerun the tests.

**** 44
      Start 44: ****_lob

44: Test command: C:\jenkins\workspace\LibSfClient-Win64-VS17-Release-Test\cmake-build-win64-VS17-Release\****s\Release\****_lob.exe
44: Working Directory: C:/jenkins/workspace/LibSfClient-Win64-VS17-Release-Test/cmake-build-win64-VS17-Release/****s
44: Test timeout computed to be: 10000000
44: [==========] Running 10 ****(s).
44: [ RUN      ] ****_lob_retrieval_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_retrieval_arrow
44: [ RUN      ] ****_lob_literal_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_literal_arrow
44: [ RUN      ] ****_lob_positional_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_positional_bind_arrow
44: [ RUN      ] ****_lob_named_bind_arrow
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_named_bind_arrow
44: [ RUN      ] ****_lob_describe_only_arrow
44: [       OK ] ****_lob_describe_only_arrow
44: [ RUN      ] ****_lob_retrieval_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_retrieval_json
44: [ RUN      ] ****_lob_literal_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_literal_json
44: [ RUN      ] ****_lob_positional_bind_json
44: Error code: 240011, message: Exceeded the retry_timeout , curl code: [28]
44: In File, C:\jenkins\workspace\LibSfClient-Win64-VS17-Release\lib\http_perform.c, Line, 586
44: [       OK ] ****_lob_positional_bind_json
44: [ RUN      ] ****_lob_named_bind_json
Build timed out (after 120 minutes). Marking the build as failed.
Build was aborted

sfc-gh-ext-simba-hx · 2025-12-23T17:42:40Z

Hi @sfc-gh-pbulawa
I think the failure on x64 is not caused by specific test case but exceeds the 2 hours timeout. I'll try reduce the time spent on lob test cases.

sfc-gh-ext-simba-hx · 2025-12-29T17:04:26Z

Hi @sfc-gh-pbulawa
Please help to check what's the latest failure on Jenkins?

sfc-gh-dprzybysz added 2 commits February 13, 2025 08:58

SNOW-1926861: Run lob tests with 128 MB limit

f062bd6

SNOW-1926861: Skip metadata verification

0ff39e7

sfc-gh-dprzybysz force-pushed the SNOW-1926861-run-lob-tests-with-128-mb-limit branch from 44c7961 to 0ff39e7 Compare February 18, 2025 09:57

sfc-gh-ext-simba-hx and others added 13 commits February 18, 2025 15:56

temporarily enable debugging

c966297

temporarily disable dump recv data

18121a0

disable dump send data as well

5e87e59

revert temporary debug changes

d17e825

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

ba858e8

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

c3f8a80

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

a77d41b

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

db01938

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

3f76909

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

1dd949f

remove debug changes

b01e3d2

fixes

15685af

fix warning

aaee789

sfc-gh-ext-simba-hx requested review from sfc-gh-dstempniak, sfc-gh-jszczerbinski and sfc-gh-pbulawa July 26, 2025 01:20

sfc-gh-ext-simba-hx marked this pull request as ready for review July 26, 2025 01:20

sfc-gh-ext-simba-hx requested a review from a team as a code owner July 26, 2025 01:20

sfc-gh-ext-simba-hx marked this pull request as draft July 28, 2025 18:41

sfc-gh-ext-simba-hx removed request for sfc-gh-dstempniak, sfc-gh-jszczerbinski and sfc-gh-pbulawa July 28, 2025 18:42

sfc-gh-ext-simba-hx added 5 commits July 28, 2025 12:00

debug lob

ab523a3

fix error with http code 307 (redirect)

33b85a9

retry on CURLE_PARTIAL_FILE

3cc5980

fix test failure due to timeout

79d1ad1

extend timeout

2dd3438

sfc-gh-ext-simba-hx and others added 4 commits December 3, 2025 09:22

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

fccb27a

extend timeout to fix timeout issue

94a3e49

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

0a15283

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

10f4db4

fix lob failures, retry on any unknown curl error

4ae783b

sfc-gh-ext-simba-hx force-pushed the SNOW-1926861-run-lob-tests-with-128-mb-limit branch from b5cdb8a to 4ae783b Compare December 8, 2025 22:15

sfc-gh-ext-simba-hx and others added 7 commits December 8, 2025 16:07

fix build warning

c16d041

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

c055fdb

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

92c515a

fix test failure

523bb5d

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

49570ef

lower the timeout to make test faster

0ab193a

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

041c1f3

sfc-gh-pbulawa approved these changes Dec 17, 2025

View reviewed changes

sfc-gh-ddas approved these changes Dec 17, 2025

View reviewed changes

sfc-gh-ext-simba-hx added 4 commits December 18, 2025 16:17

fix test failure

0a7a7e6

Merge branch 'master' into SNOW-1926861-run-lob-tests-with-128-mb-limit

9b018f0

fix test failure

44b9912

fix build error

17d752f

sfc-gh-ext-simba-hx added 4 commits December 23, 2025 09:53

shorten the lob test

636afc5

shorten lob test

6a2fe07

shorten lob test

de107bc

empty commit to rerun test

013c086

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

SNOW-1926861: Run lob tests with 128 MB limit #831

SNOW-1926861: Run lob tests with 128 MB limit #831

sfc-gh-dprzybysz commented Feb 13, 2025 •

edited by sfc-gh-ext-simba-hx

Loading

Uh oh!

sfc-gh-ext-simba-hx commented Dec 8, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 17, 2025

Uh oh!

sfc-gh-pbulawa commented Dec 18, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 22, 2025

Uh oh!

sfc-gh-pbulawa commented Dec 23, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 23, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

SNOW-1926861: Run lob tests with 128 MB limit #831

Are you sure you want to change the base?

SNOW-1926861: Run lob tests with 128 MB limit #831

Conversation

sfc-gh-dprzybysz commented Feb 13, 2025 • edited by sfc-gh-ext-simba-hx Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sfc-gh-ext-simba-hx commented Dec 8, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 17, 2025

Uh oh!

sfc-gh-pbulawa commented Dec 18, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 22, 2025

Uh oh!

sfc-gh-pbulawa commented Dec 23, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 23, 2025

Uh oh!

sfc-gh-ext-simba-hx commented Dec 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

sfc-gh-dprzybysz commented Feb 13, 2025 •

edited by sfc-gh-ext-simba-hx

Loading