One Click Installer for Windows

[nithinprabhu]

@PromtEngineer Thanks a bunch for this repo !

Inspired by one click installers provided by text-generation-webui I have created one for localGPT. I want the community members with windows PC to try it & let me know if it works for them !

localGPT_Windows.zip

[nekiee13]

Hi there, will try later this later today and then give you back feedback. Did tried to install it yesterday on my own, but all failed at building wheel for llama-cpp-python...
I've already have installed GCC and G++ Compilers and Debugger (via Msys2), so on 1st attempt I'll skip Microsoft C++ Build Tools. If it fails (again) I'll try again with 'em installed. Thank you anyway for all your effort. Hope this works. :-)

[nekiee13]

Update:-> All Ok! I had to install Microsoft C++ Build Tools, as you stated in advance. From CLI, I've got 19t/sec. Two questions

1. For Cuda device on GGML - I've changed number of off-loaded to GPU (kwargs["n_gpu_layers"]) from 1000 to 2000, but nothing changed.. How to set number of off-loaded layers to GPU?
2. For GUI: API part work ok, but python run_localGPT_UI.py does not. System send notice that streamlit run should be used, so i used it as : streamlit run localGPT_UI.py. It raise following error :

Warning: to view this Streamlit app on a browser, run it with the following
command:

streamlit run localGPT_UI.py [ARGUMENTS]

2023-08-31 01:05:41.968 Session state does not function when running a script without streamlit run
load INSTRUCTOR_Transformer
max_seq_length 512
Traceback (most recent call last):
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\session_state.py", line 380, in getitem
return self._getitem(widget_id, key)
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\session_state.py", line 425, in _getitem
raise KeyError
KeyError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\session_state_proxy.py", line 119, in getattr
return self[key]
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\session_state_proxy.py", line 90, in getitem
return get_session_state()[key]
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\safe_session_state.py", line 113, in getitem
return self._state[key]
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\session_state.py", line 382, in getitem
raise KeyError(_missing_key_error_message(key))
KeyError: 'st.session_state has no key "EMBEDDINGS". Did you forget to initialize it? More info: https://docs.streamlit.io/library/advanced-features/session-state#initialization'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\LLM\LocalGPT\localGPT\localGPT_UI.py", line 68, in
embedding_function=st.session_state.EMBEDDINGS,
File "D:\LLM\LocalGPT\installer_files\env\lib\site-packages\streamlit\runtime\state\session_state_proxy.py", line 121, in getattr
raise AttributeError(_missing_attr_error_message(key))
AttributeError: st.session_state has no attribute "EMBEDDINGS". Did you forget to initialize it? More info: https://docs.streamlit.io/library/advanced-features/session-state#initialization

Thank You for your effort. :))