关于卡顿问题

[wling-art]

在运行bot程序时，高速度的Lagrange 的反向 ws 接收信息下会让nonebot速度越来越慢，几乎是卡一会，然后同时输出几条内容，有非常严重的阻塞，我使用了cProfile来检测性能瓶颈，如下。希望得到帮助和解决方法，tks。

性能报告概览

报告 1:

Sun Jul 21 01:50:35 2024    restats

         36593804 function calls (30573139 primitive calls) in 51.279 seconds

   Ordered by: cumulative time
   List reduced from 4586 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  722/688    0.032    0.000   95.775    0.139 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py:167(greenlet_spawn)
1467/1352    0.002    0.000   90.893    0.067 {method 'acquire' of '_thread.lock' objects}
       29    0.000    0.000   90.892    3.134 /root/.pyenv/versions/3.12.3/lib/python3.12/threading.py:1115(join)
       29    0.000    0.000   90.892    3.134 /root/.pyenv/versions/3.12.3/lib/python3.12/threading.py:1153(_wait_for_tstate_lock)
    22806    0.110    0.000   67.338    0.003 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/nonebot/dependencies/__init__.py:84(__call__)
        4    0.009    0.002   59.929   14.982 {method 'run_until_complete' of 'uvloop.loop.Loop' objects}
     84/1    0.004    0.000   51.288   51.288 {built-in method builtins.exec}
        1    0.000    0.000   51.288   51.288 <string>:1(<module>)
        1    0.000    0.000   51.288   51.288 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/nonebot/__init__.py:322(run)
        1    0.000    0.000   51.288   51.288 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/nonebot/drivers/fastapi.py:159(run)

报告 2:

Sun Jul 21 02:45:41 2024    restats

         134882363 function calls (117140571 primitive calls) in 181.824 seconds

   Ordered by: cumulative time
   List reduced from 4904 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
2135/2099    0.068    0.000  855.289    0.407 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py:167(greenlet_spawn)
4201/4086    0.004    0.000  526.798    0.129 {method 'acquire' of '_thread.lock' objects}
       29    0.000    0.000  526.795   18.165 /root/.pyenv/versions/3.12.3/lib/python3.12/threading.py:1115(join)
       29    0.000    0.000  526.795   18.165 /root/.pyenv/versions/3.12.3/lib/python3.12/threading.py:1153(_wait_for_tstate_lock)
       19    0.038    0.002  408.722   21.512 {method 'throw' of 'greenlet.greenlet' objects}
    77224    0.557    0.000  301.110    0.004 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/nonebot/dependencies/__init__.py:84(__call__)
        4    0.032    0.008  232.950   58.237 {method 'run_until_complete' of 'uvloop.loop.Loop' objects}
     85/1    0.005    0.000  181.857  181.857 {built-in method builtins.exec}
        1    0.000    0.000  181.857  181.857 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/nonebot/__init__.py:322(run)
        1    0.000    0.000  181.857  181.857 /root/.cache/pypoetry/virtualenvs/u1bot-1qZ_NDF8-py3.12/lib/python3.12/site-packages/nonebot/drivers/fastapi.py:159(run)

通过 Chatgpt 分析主要发现

1. 线程锁和绿色线程:

   • acquire、join、_wait_for_tstate_lock 和 throw 的累计时间显著增加，说明线程管理和绿色线程操作可能是性能瓶颈。

2. SSL证书加载:

   • load_verify_locations 的时间较长，可能是因为证书加载频繁或证书较大。

3. Pydantic模型处理:

   • _walk、handle_model_fields_schema 和 validate_core_schema 等操作的时间增加，说明数据模型处理仍然是主要瓶颈。

4. 异步事件循环:

   • run_until_complete 和 exec 的时间显著增加，可能是因为异步任务运行时间较长或脚本执行复杂度高。

以下是我的 pyproject.toml 配置

[tool.nonebot]
adapters = [
    { name = "OneBot V11", module_name = "nonebot.adapters.onebot.v11" },
]
plugins = [
    "nonebot_plugin_datastore",
    "nonebot_plugin_apscheduler",
    "nonebot_plugin_htmlrender",
    "nonebot_plugin_alconna",
    "nonebot_plugin_memes_api",
    "nonebot_plugin_tortoise_orm",
    "nonebot_plugin_cesaa",
    "nonebot_plugin_localstore",
    "nonebot_plugin_session",
    "nonebot_plugin_saa",
    "nonebot_plugin_orm",
]
plugin_dirs = ["src/plugins"]
builtin_plugins = []


[tool.poetry.dependencies]
python = "^3.12"
nonebot-plugin-datastore = "^1.1.2"
nonebot-plugin-tortoise-orm = "^0.1.1"
nonebot-plugin-apscheduler = "^0.5.0"
nonebot-plugin-htmlrender = "^0.3.3"
pil-utils = "^0.1.10"
ujson = "^5.10.0"
httpx = "^0.27.0"
faker = "^26.0.0"
requests = "^2.31.0"
qrcode = "^7.4.2"
aiofile = "^3.8.8"
aiohttp = "^3.9.1"
pyncm = "^1.6.13"
tinydb = "^4.8.0"
jieba = "^0.42.1"
emoji = "^2.9.0"
wordcloud = "^1.9.3"
imageio = "^2.33.1"
pillow = "^10.2.0"
humanize = "^4.9.0"
psutil = "^5.9.7"
aiomysql = "^0.2.0"
sqlalchemy = "^2.0.31"
nonebot-plugin-orm = "^0.7.5"
snakeviz = "^2.2.0"
bilireq = "^0.2.12"
nonebot2 = { extras = ["quart"], version = "^2.3.2" }
starlette = "^0.37.2"
h11 = "^0.14.0"
websockets = "^12.0"
nonebot-adapter-onebot = "^2.4.3"
nonebot-plugin-eventexpiry = "^0.1.1"
jieba-fast = "^0.53"
nonebot-plugin-memes-api = "^0.3.2"
nonebot-plugin-send-anything-anywhere = "^0.6.1"
nonebot-plugin-cesaa = "^0.4.0"
nonebot-plugin-session = "^0.3.1"
nonebot-plugin-session-orm = "^0.2.0"

[yanyongyu]

请你把所有第三方插件卸载再尝试是否有卡顿。根据你的卡顿现象，我猜卡顿原因多半是插件导致的，有些插件可能存在同步处理阻塞了事件循环。卡一会儿然后一下子吐出很多条，说明nb在接收消息上远没有达到性能瓶颈，卡顿是由于处理部分导致的，请检查插件。

[wling-art]

移除 nonebot-plugin-memes 发现没问题了