Merge pull request #59 from DeepInsight-AI/new_pre

1.2.1 version

Author: LiChengxin007

README.md

@@ -38,7 +38,7 @@ https://github.com/DeepInsight-AI/DeepBI/assets/151519374/d1effbe1-5c11-4c77-86e
 3 Dashboard : Assemble persistent visualizations into dashboards.\
 4 Automated data analysis reports (to be developed) : Complete data analysis reports automatically according to user instructions.\
 5 Support multiple data sources, including MySQL, PostgreSQL, Doris, StarRocks, CSV/Excel, etc.\
-6 Multi-platform support, support Windows-WSL, Linux, Mac. \
+6 Multi-platform support, support Windows-WSL,Windows, Linux, Mac. \
 7 International, support Chinese, English.
 
 
@@ -104,13 +104,14 @@ git clone http://github.com/DeepInsight-AI/DeepBI.git
 
 
 ## Contact Us
-<a><img src="https://github.com/DeepInsight-AI/DeepBI/assets/151519374/b0ba1fc3-8c71-4bf7-bd53-ecf17050581a" width="40%"/></a>
+<a><img src="https://github.com/DeepInsight-AI/DeepBI/assets/151519915/1b7d9848-49fb-4f06-bad7-86d751841a6b" width="40%"/></a>
 
 
 
 ## 📑 Other
 - We have tested on Mac OS 12.7/13.X /14.1.1, Ubuntu 20.04/22.04, and Windows11 WSL 22.04.
 - Windows 10 requires version 22H2 or higher to install WSL
+- The minimum memory requirement for server operation is 1 core 2G memory, and 2 core 4G memory is recommended
 - If you have any question, please contact us at dev@deep-insight.co
 - <a href="https://github.com/DeepInsight-AI/DeepBI/issues">Issue</a>
 

README_CN.md

@@ -2,7 +2,7 @@
 
 <div align="center">
 
-DeepBi是一款AI原生的数据分析平台。DeepBI充分利用大语言模型的能力来探索、查询、可视化和共享来自任何数据源的数据。用户可以使用DeepBI洞察数据并做出数据驱动的决策。
+DeepBI是一款AI原生的数据分析平台。DeepBI充分利用大语言模型的能力来探索、查询、可视化和共享来自任何数据源的数据。用户可以使用DeepBI洞察数据并做出数据驱动的决策。
 
 
 </div>
@@ -44,7 +44,7 @@ https://github.com/DeepInsight-AI/DeepBI/assets/151519374/f0d2fcd4-32b0-4095-a89
 3 仪表板大屏：将持久化的可视化图组装为仪表板。\
 4 自动化数据分析报告（待开发)：根据用户指令自动完成完整的数据分析报告。\
 5 多数据源支持，支持 MySQL、PostgreSQL、Doris，Starrocks, CSV/Excel等。\
-6 多平台支持，支持 Windows-WSL、Linux、Mac。\
+6 多平台支持，支持 Windows-WSL、Windows、Linux、Mac。\
 7 国际化，支持中文、英文。
 
 
@@ -101,6 +101,7 @@ git clone http://github.com/DeepInsight-AI/DeepBI.git
 ## 📑 文档
 - 我们已经在 Mac OS 12.7/13.X /14.1.1 , Ubuntu 20.04/22.04 和  Windows11 WSL 22.04 系统测试。
 - Windows 10 安装WSL需要 22H2版本，详见  [使用 Docker 安装](Docker_install_CN.md)
+- 服务器运行需求 最低 1核心 2G内存，建议2核心 4G内存 以上
 - 如果有什么问题，可以联系我们 dev@deep-insight.co
 - <a href="https://github.com/DeepInsight-AI/DeepBI/issues">Issue</a>
 

README_window_cn.md

@@ -25,3 +25,55 @@
  ![docker_err.jpg](user_manual/cn/exe/12.jpg)
 
 - 以上就是 DeepBI安装过程，使用说明，请点击 [DeepBI 在线使用说明](client/app/assets/images/cn/user_manual_cn.md)
+
+## Exe安装遇到问题解决方案
+
+1. 安装过程提示没有权限 见下图：：
+ ![docker err.jpg](user_manual/en/exe/13.png)
+- Appear：
+```
+The files belonging to this database system will be owned by user "00767127".
+This user must also own the server process. The database cluster will be initialized with locale "Chinese (Simplified)_China.936".
+ initdb: could not find suitable text search configuration for locale "Chinese (Simplified)_China.936"
+  The default text search configuration will be set to "simple".
+ Data page checksums are disabled. fixing permissions on existing directory C:/DeepBI/pgsqldata ...
+ initdb: error: could not change permissions of directory "C:/DeepBI/pgsqldata": Permission denied pg_ctl: directory "C:/DeepBI/pgsqldata" is not a database
+ cluster directory createdb: error: connection to server at "localhost" (::1), port 5432 failed: Connection refused (0x0000274D/10061) Is the server running on that host and accepting TCP/IP connections?
+```
+- solution：
+
+（1）（1）停止服务，打开命令行，以管理员身份打开,点击window 窗口，直接输入cmd 如下图:
+
+![docker err.jpg](user_manual/cn/exe/14.png)
+
+（2）上面命令命令一般再C:\WINDOWS\system32,转到C:DeepBI 文件夹下，运行 stop.bat
+
+ 运行: ``` cd ../../DeepBI```
+ 运行: ```stop.bat``` ,等待运行完毕
+
+![docker err.jpg](user_manual/cn/exe/15.png)
+
+ （3）删除历史无权限文件
+
+  打开 C:\DeepBI 然后删除 .init文件 和 pgsqldata 文件夹
+
+ （4）给文件夹权限
+
+![docker err.jpg](user_manual/cn/exe/16.png)
+
+ （5）重新打开管理员命令行，到DeepBI下 命令行运行start.bat 等待启动即可
+
+2. 安装完毕设置网页账号登录后一直处于加载中。
+
+![docker err.jpg](user_manual/cn/exe/17.png)
+
+ 右击页面，选择检查，选择
+
+![docker err.jpg](user_manual/cn/exe/18.png)
+
+如果出现js无法加载
+
+解决方法：
+
+![docker err.jpg](user_manual/cn/exe/18-1.png)
+

README_window_en.md

@@ -25,3 +25,56 @@
  ![docker err.jpg](user_manual/en/exe/12.jpg)
 - Above is DeepBI installation process, use instructions, [DeepBI user manual](client/app/assets/images/en/user_manual_en.md)
 
+## An error occurred during installation and the solution
+
+1. Promblem: A message indicating no permission is displayed during installation, as fellows：
+ ![docker err.jpg](user_manual/en/exe/13.png)
+- Appear：
+```
+The files belonging to this database system will be owned by user "00767127".
+This user must also own the server process. The database cluster will be initialized with locale "Chinese (Simplified)_China.936".
+ initdb: could not find suitable text search configuration for locale "Chinese (Simplified)_China.936"
+  The default text search configuration will be set to "simple".
+ Data page checksums are disabled. fixing permissions on existing directory C:/DeepBI/pgsqldata ...
+ initdb: error: could not change permissions of directory "C:/DeepBI/pgsqldata": Permission denied pg_ctl: directory "C:/DeepBI/pgsqldata" is not a database
+ cluster directory createdb: error: connection to server at "localhost" (::1), port 5432 failed: Connection refused (0x0000274D/10061) Is the server running on that host and accepting TCP/IP connections?
+```
+- solution：
+
+（1）Stop the service and open the command line as an administrator
+  Click the Window window and enter cmd as shown below:
+
+![docker err.jpg](user_manual/en/exe/14.png)
+
+（2）The command after the above command is generally in the directory 	C:\WINDOWS\system32,Go to the C:DeepBI folder and run stop.bat
+ run command: ``` cd ../../DeepBI```
+ run command: ```stop.bat``` ,then wait for the run to finish
+
+![docker err.jpg](user_manual/en/exe/15.png)
+
+ （3） Delete historical files without permission
+    Open folder C:\DeepBI and delete the “.init” file and the “pgsqldata” folder
+
+ （4）Give folder permission
+
+![docker err.jpg](user_manual/en/exe/16.png)
+
+ （5）Reopen the administrator command line cmd and go to C:\DeepBI
+Run start.bat on the CLI and wait until the command is complete
+
+2. Promblem: After the installation is complete, the web account is keeping loading after login.
+
+![docker err.jpg](user_manual/en/exe/17.png)
+
+ Right-click the page, select Check, and select Console
+
+![docker err.jpg](user_manual/en/exe/18.png)
+
+If the js files can not load
+
+Solution：
+
+（1）WIN+R Open the CMD command box and enter "regedit" to enter the registry
+
+（2）In the "Computer /HKEY_CLASSES_ROOT/.js" directory, locate the Content Type key and change its value "text/plain" to "text/javascript"
+

ai/agents/agent_instance_util.py

@@ -4,16 +4,22 @@
 from ai.backend.util.write_log import logger
 import traceback
 from ai.backend.util.token_util import num_tokens_from_messages
-from ai.backend.base_config import request_timeout, max_retry_times, language_chinese, \
-    language_english, \
-    local_base_postgresql_info, local_base_mysql_info, local_base_xls_info, \
-    csv_file_path, python_base_dependency, default_language_mode
 from ai.agents.prompt import CSV_ECHART_TIPS_MESS, \
     MYSQL_ECHART_TIPS_MESS, MYSQL_MATPLOTLIB_TIPS_MESS, POSTGRESQL_ECHART_TIPS_MESS
 from ai.agents.agentchat import (UserProxyAgent, GroupChat, AssistantAgent, GroupChatManager,
                                  PythonProxyAgent, BIProxyAgent, TaskPlannerAgent, TaskSelectorAgent, CheckAgent,
                                  ChartPresenterAgent)
-
+from ai.backend.base_config import CONFIG
+
+max_retry_times = CONFIG.max_retry_times
+language_chinese = CONFIG.language_chinese
+language_english = CONFIG.language_chinese
+default_language_mode = CONFIG.default_language_mode
+local_base_postgresql_info = CONFIG.local_base_postgresql_info
+local_base_xls_info = CONFIG.local_base_xls_info
+local_base_mysql_info = CONFIG.local_base_mysql_info
+python_base_dependency = CONFIG.python_base_dependency
+request_timeout = CONFIG.request_timeout
 
 class AgentInstanceUtil:
     def __init__(
@@ -72,7 +78,7 @@ def set_api_key(self, api_key, api_host=None):
                     'model': 'gpt-4-1106-preview',
                     'api_key': self.api_key,
                     'api_base': api_host,
-                    'api_type': 'azure',
+
                 },
             ]
 
@@ -1072,7 +1078,7 @@ def set_base_csv_info(self, db_info):
 
                 tb_desc = {
                     "file_comment": table_comment,
-                    "file_path": csv_file_path + table_name,
+                    "file_path": CONFIG.csv_file_path + table_name,
                 }
                 print('tb_desc : ', tb_desc)
                 csv_content.append(tb_desc)

ai/agents/agentchat/bi_proxy_agent.py

@@ -17,7 +17,6 @@
 from ai.backend.util.write_log import logger
 from ai.backend.util.token_util import num_tokens_from_messages
 import traceback
-from ai.backend.base_config import if_hide_sensitive
 
 try:
     from termcolor import colored

ai/agents/code_utils.py

@@ -11,8 +11,10 @@
 from ai.agents import oai
 import json
 import traceback
-from ai.backend.base_config import if_hide_sensitive
+from ai.backend.base_config import CONFIG
 
+
+if_hide_sensitive = CONFIG.if_hide_sensitive
 try:
     import docker
 except ImportError:

ai/backend/aidb/aidb.py

@@ -10,6 +10,8 @@
 import time
 from ai.backend.util import base_util
 import asyncio
+from requests.exceptions import HTTPError
+from ai.backend.language_info import LanguageInfo
 
 
 class AIDB:
@@ -59,6 +61,11 @@ async def get_data_desc(self, q_str):
             )
             answer_message = planner_user.last_message()["content"]
             # print("answer_message: ", answer_message)
+        except HTTPError as http_err:
+            traceback.print_exc()
+            error_message = self.generate_error_message(http_err)
+            return await self.put_message(500, CONFIG.talker_bi, CONFIG.type_comment_first, error_message)
+
         except Exception as e:
             traceback.print_exc()
             logger.error("from user:[{}".format(self.user_name) + "] , " + "error: " + str(e))
@@ -132,8 +139,8 @@ async def check_data_base(self, q_str):
 
                         qustion_message = """Help me check that the following data comments are complete and correct."""
 
-                        if self.language_mode == CONFIG.language_chinese:
-                            qustion_message = "帮助我检查下列数据注释是否完整且正确: "
+                        # if self.language_mode == CONFIG.language_chinese:
+                        #     qustion_message = "帮助我检查下列数据注释是否完整且正确: "
 
                         await asyncio.wait_for(planner_user.initiate_chat(
                             database_describer,
@@ -225,9 +232,12 @@ async def put_message(self, state=200, receiver='log', data_type=None, content=N
         # await self.ws.send(consume_output)
         if self.websocket is not None:
             await asyncio.wait_for(self.websocket.send(consume_output), timeout=CONFIG.request_timeout)
-        print(str(time.strftime("%Y-%m-%d %H:%M:%S",
-                                time.localtime())) + ' ---- ' + "from user:[{}".format(
-            self.user_name) + "], reply a message:{}".format(consume_output))
+
+        send_mess = str(time.strftime("%Y-%m-%d %H:%M:%S",
+                                      time.localtime())) + ' ---- ' + "from user:[{}".format(
+            self.user_name) + "], reply a message:{}".format(consume_output)
+        print(send_mess)
+        logger.info(send_mess)
 
     async def check_api_key(self):
         # self.agent_instance_util.api_key_use = True
@@ -259,6 +269,12 @@ async def check_api_key(self):
                 self.agent_instance_util.api_key_use = True
 
                 return True
+            except HTTPError as http_err:
+                traceback.print_exc()
+
+                error_miss_key = self.generate_error_message(http_err, error_message=LanguageInfo.api_key_fail)
+                await self.put_message(500, CONFIG.talker_log, CONFIG.type_log_data, error_miss_key)
+                return False
             except Exception as e:
                 traceback.print_exc()
                 logger.error("from user:[{}".format(self.user_name) + "] , " + "error: " + str(e))
@@ -281,8 +297,7 @@ async def test_api_key(self):
             try:
                 ApiKey, HttpProxyHost, HttpProxyPort, ApiHost = self.load_api_key(token_path)
                 if ApiKey is None or len(ApiKey) == 0:
-                    await self.put_message(500, CONFIG.talker_log, CONFIG.type_log_data, self.error_miss_key)
-                    return False
+                    return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, LanguageInfo.no_api_key)
 
                 self.agent_instance_util.set_api_key(ApiKey, ApiHost)
 
@@ -301,28 +316,21 @@ async def test_api_key(self):
 
                 self.agent_instance_util.api_key_use = True
 
-                if self.language_mode == CONFIG.language_chinese:
-                    return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, '检测通过')
-                else:
-                    return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, 'test success')
+                return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, LanguageInfo.api_key_success)
 
+            except HTTPError as http_err:
+                traceback.print_exc()
 
+                error_miss_key = self.generate_error_message(http_err, error_message=LanguageInfo.api_key_fail)
+                return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, error_miss_key)
 
             except Exception as e:
                 traceback.print_exc()
                 logger.error("from user:[{}".format(self.user_name) + "] , " + "error: " + str(e))
-
-                if self.language_mode == CONFIG.language_chinese:
-                    return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, '检测未通过...')
-                else:
-                    return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, 'test fail')
+                return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, LanguageInfo.api_key_fail)
 
         else:
-            if self.language_mode == CONFIG.language_chinese:
-                return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, '未检测到apikey,请先保存')
-            else:
-                return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test,
-                                              'apikey not detected, please save first')
+            return await self.put_message(200, CONFIG.talker_api, CONFIG.type_test, LanguageInfo.no_api_key)
 
     def load_api_key(self, token_path):
         ApiKey = None
@@ -349,7 +357,8 @@ def load_api_key(self, token_path):
             elif in_use == 'DeepInsight':
                 ApiKey = data[in_use]['ApiKey']
                 print('DeepBIApiKey : ', ApiKey)
-                ApiHost = "https://apiserver.deep-thought.io/proxy"
+                # ApiHost = "https://apiserver.deep-thought.io/proxy"
+                ApiHost = CONFIG.ApiHost
         else:
             ApiKey = data['OpenaiApiKey']
             print('OpenaiApiKey : ', ApiKey)
@@ -359,3 +368,27 @@ def load_api_key(self, token_path):
             print('HttpProxyPort : ', HttpProxyPort)
 
         return ApiKey, HttpProxyHost, HttpProxyPort, ApiHost
+
+    def generate_error_message(self, http_err, error_message=' API ERROR '):
+        # print(f'HTTP error occurred: {http_err}')
+        # print(f'Response status code: {http_err.response.status_code}')
+        # print(f'Response text: {http_err.response.text}')
+
+        # error_message = self.error_miss_key
+        status_code = http_err.response.status_code
+        if str(http_err.response.text).__contains__('deep-thought'):
+            if status_code == 401:
+                error_message = error_message + str(status_code) + ' , APIKEY Empty Error'
+            elif status_code == 402:
+                error_message = error_message + str(status_code) + ' , Data Empty Error'
+            elif status_code == 403:
+                error_message = error_message + str(status_code) + ' , APIKEY Error'
+            elif status_code == 404:
+                error_message = error_message + str(status_code) + ' , Unsupported Ai Engine Error'
+            elif status_code == 405:
+                error_message = error_message + str(status_code) + ' , Insufficient Token Error'
+            elif status_code == 500:
+                error_message = error_message + str(status_code) + ' , OpenAI API Error, ' + str(http_err.response.text)
+        else:
+            error_message = error_message + ' ' + str(status_code) + ' ' + str(http_err.response.text)
+        return error_message