sdl-mcp index failed

[cheuk-cheng]

There are more than 10000 files in this project directory below.

$ sdl-mcp index -c sdlmcp.config.json

hanged for a long time so I press ctrl-C to abort it.

Try to index again (incremental mode)

$ sdl-mcp index -c sdlmcp.config.json

pass1:
...
pass2: 55/57 prx/internal/tui/style/style.go
pass2: 56/57 prx/internal/tui/tui.go
pass2: 57/57

<--- Last few GCs --->

[9196:0xa7c000000] 2866171 ms: Scavenge (during sweeping) 4076.6 (4101.5) -> 4075.9 (4106.5) MB, pooled: 0.0 MB, 21.07 / 0.01 ms (average mu = 0.472, current mu = 0.245) allocation failure;
[9196:0xa7c000000] 2868215 ms: Mark-Compact (reduce) 4082.7 (4107.8) -> 4080.9 (4099.0) MB, pooled: 0.0 MB, 927.15 / 0.01 ms (+ 575.2 ms in 106 steps since start of marking, biggest step 27.5 ms, walltime since start of marking 1545 ms) (average mu = 0.4
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

1: 0x1090ac274 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
2: 0x1093bf6f8 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
3: 0x1093bf6b0 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
4: 0x10956b93c v8::internal::Heap::ShouldOptimizeForLoadTime() const [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
5: 0x10956aff8 v8::internal::Heap::EagerlyFreeExternalMemoryAndWasmCode() [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
6: 0x10956a18c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags, v8::internal::PerformHeapLimitCheck) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
7: 0x1094fa09c v8::internal::StackGuard::HandleInterrupts(v8::internal::StackGuard::InterruptLevel) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
8: 0x109854c28 v8::internal::Runtime_HandleNoHeapWritesInterrupts(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
9: 0x108ecd474 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
10: 0x12b28c6b4
11: 0x12b3ad3f4
12: 0x12b3825b4
13: 0x108e6b4c4 Builtins_AsyncFunctionAwaitResolveClosure [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
14: 0x108f42058 Builtins_PromiseFulfillReactionJob [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
15: 0x108e5a920 Builtins_RunMicrotasks [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
16: 0x108e2a850 Builtins_JSRunMicrotasksEntry [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
17: 0x1094d6990 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
18: 0x1094d7040 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
19: 0x1094d7130 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
20: 0x1094f8620 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
21: 0x1094f843c v8::internal::MicrotaskQueue::PerformCheckpointInternal(v8::Isolate*) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
22: 0x108e2f298 Builtins_CallApiCallbackGeneric [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
23: 0x12ae161b4
24: 0x108e2a96c Builtins_JSEntryTrampoline [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
25: 0x108e2a610 Builtins_JSEntry [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
26: 0x1094d69c0 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
27: 0x1094d63a4 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::DirectHandlev8::internal::Object, v8::internal::DirectHandlev8::internal::Object, v8::base::Vector<v8::internal::DirectHandlev8::internal::Object const>) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
28: 0x10a0e4334 v8::Function::Call(v8::Isolate*, v8::Localv8::Context, v8::Localv8::Value, int, v8::Localv8::Value) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
29: 0x109005924 node::InternalCallbackScope::Close() [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
30: 0x109005b9c node::InternalMakeCallback(node::Environment, v8::Localv8::Object, v8::Localv8::Object, v8::Localv8::Function, int, v8::Localv8::Value, node::async_context, v8::Localv8::Value) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
31: 0x109014994 node::AsyncWrap::MakeCallback(v8::Localv8::Function, int, v8::Localv8::Value) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
32: 0x109171a00 node::(anonymous namespace)::ProcessWrap::OnExit(uv_process_s*, long long, int) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
33: 0x104c9d608 uv__wait_children [/opt/homebrew/Cellar/libuv/1.52.1/lib/libuv.1.0.0.dylib]
34: 0x104ca520c uv__io_poll [/opt/homebrew/Cellar/libuv/1.52.1/lib/libuv.1.0.0.dylib]
35: 0x104c96b64 uv_run [/opt/homebrew/Cellar/libuv/1.52.1/lib/libuv.1.0.0.dylib]
36: 0x1090068cc node::SpinEventLoopInternal(node::Environment*) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
37: 0x1090da618 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
38: 0x1090da34c node::NodeMainInstance::Run() [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
39: 0x10907e650 node::Start(int, char**) [/opt/homebrew/Cellar/node/25.6.1/lib/libnode.141.dylib]
40: 0x1894f5d54 start [/usr/lib/dyld]
zsh: abort sdl-mcp index -c sdlmcp.config.json

[GlitterKill]

which engine was this? Rust or Typescript? Let me know, and try the other engine.

[GlitterKill]

It actually may not be hung when it appears to be. I need to have it give you a progress after the pass2 index when it is calculating all the relationships. With 10000 files it may take quite a while. I plan to add a high performance indexer next for large repos like that. Just need to clean up old stuff first.

[cheuk-cheng]

Thanks @GlitterKill. I was indexing https://github.com/metabase/metabase repo above. Also, I did not press ctrl-C (a second time) when I ran sdl-mcp index after I aborted it one time and after the above errors were printed, it exited back to the shell after showing zsh: abort sdl-mcp index -c sdlmcp.config.json message.