Webdriveragent freeze on simulators when running in parallel

[Petrdontone]

Describe the bug

I'm running tests in 8 threads on a Mac Studio M4 Max 36GB on simulators. When I launch them, I see that on some devices, it takes a long time to launch, say, 3-4 minutes. Overall, I see that the tests are running on all simulators, with tests generally running on 3-4 simulators, while the others are "resting" and only launching on them after a long time.
Looking at the logs, it seems there's a problem with the webdriver agent build.
I also noticed that there's a derivedDataPath parameter when running, for example, "derivedDataPath":"/Users/virtual_farm_ios/Library/Developer/Xcode/DerivedData/WebDriverAgent-7EACCBD0-58FF-4BE9-8818-D6F25EBCF2CB"
but when running xcodebuild, the derivedDataPath isn't set.
"-derivedDataPath /Users/virtual_farm_ios/.appium/node_modules/appium-xcuitest-driver/node_modules"
@saikrishna321 @sudharsan-selvaraj help pls

I have attached all the capabilities and much more in the log file.

Link to Appium logs

It won't attach to GitHub, I can only do it as a file.

loggggggsss.txt.rtf

Environment

1. Appium version (or git revision) that exhibits the issue: 3.1.0 (Appium), 11.2.0 (Appium device farm)
2. Last Appium version that did not exhibit the issue (if applicable):
3. Node.js version (unless using Appium.app|exe): 20.19.0
4. Npm or Yarn package manager: npm
5. Mobile platform/version under test: iOS
6. Real device or emulator/simulator: Simulator
7. Appium CLI or Appium.app|exe: Appium CLI

[Petrdontone]

@saikrishna321 @sudharsan-selvaraj helppp please))

[Petrdontone]

@saikrishna321 @sudharsan-selvaraj helpppppp please)

[saikrishna321]

@Petrdontone will look into this.

[Petrdontone]

@saikrishna321 Please tell me, did you watch it?)

[Petrdontone]

@saikrishna321 up)

[Petrdontone]

@saikrishna321 I also see this message in the logs
[0a254c37][XCUITestDriver@bd91] Starting WebDriverAgent initialization with the synchronization key 'XCUITestDriver'
[0a254c37][XCUITestDriver@bd91] Consider setting a unique 'derivedDataPath' capability value for each parallel driver instance to avoid conflicts and speed up the building process

[saikrishna321]

Please try latest this is fixed

[Petrdontone]

@saikrishna321
I tried it on the new version, and it seemed to improve, but I noticed a different problem.
At one point, my tests froze.
I stopped the appium server, ran command
ps aux | grep node
and saw that the xcode command was running in the processes, even though the tests weren't running. I killed all the processes, ran the tests again, and the next time, the tests ran very quickly, but after the tests finished, I still saw xcode commands.

And some additional information: when my tests froze for the first time, I saw this in the logs:
(node:973) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 attached listeners added to [Socket]. MaxListeners is 10. Use emitter.setMaxListeners() to increase the limit.

UPD(2):

After running the appium command in the terminal for some time, an exception is thrown and I see the following error:

<--- Last few GCs --->

[52650:0x9be800000]  9392797 ms: Mark-Compact 4046.8 (4136.8) -> 4034.8 (4141.0) MB, 378.46 / 0.00 ms  (average mu = 0.878, current mu = 0.474) allocation failure; scavenge might not succeed
[52650:0x9be800000]  9393750 ms: Mark-Compact 4050.6 (4141.0) -> 4039.1 (4145.3) MB, 850.75 / 0.00 ms  (average mu = 0.674, current mu = 0.107) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0x100c31334 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 2: 0x100dede84 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 3: 0x100ffbcd8 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 4: 0x100ffa640 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 5: 0x100ff04dc v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 6: 0x100ff0d34 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 7: 0x100fd5214 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 8: 0x1013e234c v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
 9: 0x101774c44 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
10: 0x10b62f95c 
11: 0x10b108360 
12: 0x10b4181d0 
13: 0x1017cf524 Builtins_PromiseConstructor [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
14: 0x1016e98fc Builtins_JSBuiltinsConstructStub [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
15: 0x10b65dc7c 
16: 0x10b672a78 
17: 0x101723210 Builtins_AsyncFunctionAwaitResolveClosure [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
18: 0x1017d0fb8 Builtins_PromiseFulfillReactionJob [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
19: 0x101712b94 Builtins_RunMicrotasks [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
20: 0x1016ea3f4 Builtins_JSRunMicrotasksEntry [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
21: 0x100f66748 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
22: 0x100f67074 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
23: 0x100f67260 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
24: 0x100f91f50 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
25: 0x100f926dc v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
26: 0x1016edef0 Builtins_CallApiCallback [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
27: 0x10b419268 
28: 0x1016ea50c Builtins_JSEntryTrampoline [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
29: 0x1016ea1f4 Builtins_JSEntry [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
30: 0x100f66790 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
31: 0x100f66024 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
32: 0x100e09eb4 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
33: 0x100b48d5c node::InternalCallbackScope::Close() [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
34: 0x100b48fb4 node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
35: 0x100b492d4 node::MakeCallback(v8::Isolate*, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
36: 0x100bc7bbc node::Environment::CheckImmediate(uv_check_s*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
37: 0x1016d2820 uv__run_check [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
38: 0x1016cc114 uv_run [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
39: 0x100b497e8 node::SpinEventLoopInternal(node::Environment*) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
40: 0x100c7448c node::NodeMainInstance::Run() [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
41: 0x100bf3a78 node::Start(int, char**) [/Users/virtual_farm_ios/.nvm/versions/node/v20.19.0/bin/node]
42: 0x194155d54 start [/usr/lib/dyld]
zsh: abort      appium server --use-plugins=device-farm --relaxed-security --allow-cors  

[Petrdontone]

@saikrishna321 Please help, I've added new information)

[Petrdontone]

again this error zsh: abort appium server --use-plugins=device-farm --relaxed-security --allow-cors
full_logs

full_full_loggss.txt

[Petrdontone]

@saikrishna321 additional logs

additional_logs.txt

[Petrdontone]

There is no such problem in version 11.2.4, but there is in 11.2.6

[saikrishna321]

We have not done any changes related to this, parallel tests and node memory is purely on ur machine usage.

[Petrdontone]

@saikrishna321
Couldn't this be because too many ports are being allocated?
There's a bug on iOS #1614
it calls the "Port has been allocated" command approximately every 5-10 seconds.

[device-farm-main] tvOS Simulators: false
[device-farm-main] No tvOS simulators found!
[device-farm-main] Port 53797 has been allocated
[device-farm-main] Port 53798 has been allocated
[device-farm-main] Port 53799 has been allocated
[device-farm-main] Port 53800 has been allocated
[device-farm-main] Port 53801 has been allocated
[device-farm-main] Port 53802 has been allocated
[device-farm-main] Port 53803 has been allocated
[device-farm-main] Port 53804 has been allocated
[device-farm-main] Port 53805 has been allocated
[device-farm-main] Port 53806 has been allocated
[device-farm-main] Port 53807 has been allocated
[device-farm-main] Port 53808 has been allocated
[device-farm-main] Port 53809 has been allocated
[device-farm-main] Port 53810 has been allocated
[device-farm-main] Updating Hub with Simulators
[device-farm-main] Updating remote android devices http://10.123.16.252:4444/device-farm/api/register
[device-farm-main] Pushed devices to hub 7EACCBD0-58FF-4BE9-8818-D6F25EBCF2CB 53810
[device-farm-main] Pushed devices to hub 3B582B61-2B8C-459F-98DE-022918E38F81 53808
[device-farm-main] Pushed devices to hub 9A6FB1D5-F14A-42DE-AA07-E92CBF35346C 53806
[device-farm-main] Pushed devices to hub 1DABE29A-BC4C-4AB1-9538-E5F844E8D6B9 53804
[device-farm-main] Pushed devices to hub 0121FC35-1361-40E8-A966-14F1B9C27C2B 53802
[device-farm-main] Pushed devices to hub C2D3A5EB-16E5-445A-B4FB-BD42829BF9E5 53800
[device-farm-main] Pushed devices to hub F34E74DA-7A13-4215-857B-7C590EF33A4C 53798
[device-farm-main] Fetching local simulators
[HTTP] --> GET /device-farm/api/status
[HTTP] <-- GET /device-farm/api/status 200 0 ms - 66
[device-farm-main] iOS Simulators: true
[device-farm-main] tvOS Simulators: false
[device-farm-main] No tvOS simulators found!
[device-farm-main] Port 53812 has been allocated
[device-farm-main] Port 53813 has been allocated
[device-farm-main] Port 53814 has been allocated
[device-farm-main] Port 53815 has been allocated
[device-farm-main] Port 53816 has been allocated
[device-farm-main] Port 53817 has been allocated
[device-farm-main] Port 53818 has been allocated
[device-farm-main] Port 53819 has been allocated
[device-farm-main] Port 53820 has been allocated
[device-farm-main] Port 53821 has been allocated
[device-farm-main] Port 53822 has been allocated
[device-farm-main] Port 53823 has been allocated
[device-farm-main] Port 53824 has been allocated
[device-farm-main] Port 53825 has been allocated