Running localGPT in WSL - crashes?

[achillez]

All the steps work fine but then on this last stage:

python3 run_localGPT.py

It always "kills" itself. Doesn't matter if I use GPU or CPU version. Any advice on this?

thanks

--

Running on: cuda
load INSTRUCTOR_Transformer
max_seq_length 512
Using embedded DuckDB with persistence: data will be stored in: /home/achillez/devel/localGPT/DB
Downloading tokenizer.model: 100%|██████████████████████████████████████████████████| 500k/500k [00:00<00:00, 27.8MB/s]
Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████| 411/411 [00:00<00:00, 4.74MB/s]
Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████| 715/715 [00:00<00:00, 9.55MB/s]
Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████| 582/582 [00:00<00:00, 6.48MB/s]
Downloading (…)model.bin.index.json: 100%|█████████████████████████████████████████| 26.8k/26.8k [00:00<00:00, 174MB/s]
Downloading (…)l-00001-of-00002.bin: 100%|████████████████████████████████████████| 9.98G/9.98G [01:44<00:00, 95.7MB/s]
Downloading (…)l-00002-of-00002.bin: 100%|████████████████████████████████████████| 3.50G/3.50G [00:36<00:00, 96.8MB/s]
Downloading shards: 100%|████████████████████████████████████████████████████████████████| 2/2 [02:20<00:00, 70.32s/it]
Killed

[PromtEngineer]

I have not been able to replicate this on my end but this seems to be a common issue. Can you share your PC Configuration? I can suggest two possible things to test.

1. Run a different embedding model: please see here: https://github.com/HKUNLP/instructor-embedding#model-list
2. Try a quantized version of Vicuna-7B instead.

Let me know if any of this helps.

[yassirscreed]

Same issue here, running on WSL-2 followed every step but at the ingest.py it gave me this Loading documents from /home/yassirscreed/GPTFun/localGPT/SOURCE_DOCUMENTS Loaded 1 documents from /home/yassirscreed/GPTFun/localGPT/SOURCE_DOCUMENTS Split into 72 chunks of text Downloading (…)7f436/.gitattributes: 100%|█████████████████████████████████████████| 1.48k/1.48k [00:00<00:00, 14.5MB/s] Downloading (…)_Pooling/config.json: 100%|█████████████████████████████████████████████| 270/270 [00:00<00:00, 2.93MB/s] Downloading (…)/2_Dense/config.json: 100%|█████████████████████████████████████████████| 116/116 [00:00<00:00, 1.45MB/s] Downloading pytorch_model.bin: 100%|███████████████████████████████████████████████| 3.15M/3.15M [00:00<00:00, 14.4MB/s] Downloading (…)0daf57f436/README.md: 100%|██████████████████████████████████████████| 66.3k/66.3k [00:00<00:00, 658kB/s] Downloading (…)af57f436/config.json: 100%|█████████████████████████████████████████| 1.52k/1.52k [00:00<00:00, 17.6MB/s] Downloading (…)ce_transformers.json: 100%|█████████████████████████████████████████████| 122/122 [00:00<00:00, 1.45MB/s] Downloading pytorch_model.bin: 100%|███████████████████████████████████████████████| 4.96G/4.96G [02:59<00:00, 27.7MB/s] Downloading (…)nce_bert_config.json: 100%|████████████████████████████████████████████| 53.0/53.0 [00:00<00:00, 642kB/s] Downloading (…)cial_tokens_map.json: 100%|█████████████████████████████████████████| 2.20k/2.20k [00:00<00:00, 24.0MB/s] Downloading spiece.model: 100%|██████████████████████████████████████████████████████| 792k/792k [00:00<00:00, 43.6MB/s] Downloading (…)7f436/tokenizer.json: 100%|█████████████████████████████████████████| 2.42M/2.42M [00:00<00:00, 12.0MB/s] Downloading (…)okenizer_config.json: 100%|█████████████████████████████████████████| 2.40k/2.40k [00:00<00:00, 27.7MB/s] Downloading (…)f57f436/modules.json: 100%|█████████████████████████████████████████████| 461/461 [00:00<00:00, 5.04MB/s] load INSTRUCTOR_Transformer Killed

Tried running it again but ended up with same error
Loading documents from /home/yassirscreed/GPTFun/localGPT/SOURCE_DOCUMENTS Loaded 1 documents from /home/yassirscreed/GPTFun/localGPT/SOURCE_DOCUMENTS Split into 72 chunks of text load INSTRUCTOR_Transformer Killed

Running on a 3700x with Vega 64, 16gb of RAM, tried both cpu and gpu versions but no success.