add globals.config and OnlineChat support dynamic source#1067
add globals.config and OnlineChat support dynamic source#1067wzh1994 merged 17 commits intoLazyAGI:mainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a robust global configuration mechanism and enables dynamic selection of online LLM sources, significantly improving the system's adaptability. It also expands the core file system abstraction with new copy and move functionalities across multiple cloud providers, accompanied by extensive documentation. These changes collectively enhance the framework's flexibility, configurability, and file management capabilities. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
这个 PR 包含两个主要的功能增强：
- 为
OnlineChat模块增加了动态数据源（dynamic source）的支持，并引入了globals.config来实现请求级别的配置覆盖。 - 为文件系统抽象（
LazyLLMFSBase）添加了copy和move操作，并为 S3、Google Drive、Feishu 等多个后端实现了这些操作。
此外，代码进行了一些重构，例如将 StaticParams 的处理逻辑统一移动到了 LLMBase 中，提高了代码的可维护性。
整体来看，这是一个重要的功能更新。我发现了一些可以改进的地方，主要涉及动态数据源的认证处理、全局配置中的一个潜在 bug，S3 后端 move 操作的性能问题，以及一个与 Python 内置 type 关键字冲突的参数命名。请查看具体的审查评论。
lazyllm/common/globals.py
Outdated
| assert __key in self._supported_configs, f'Config {__key} is not supported' | ||
| return globals['config'].get(__key) or config[__key] |
There was a problem hiding this comment.
__getitem__ 的实现中存在一个潜在的 bug。当 globals['config'].get(__key) 返回一个 "falsy" 值（如 False, 0, ''）时，or 表达式会错误地回退到 config[__key]，导致无法通过 globals 正确地覆盖这些值。
例如，如果全局配置中一个键的值是 True，而在某个请求中希望通过 globals 将其覆盖为 False，当前实现会失败。
建议修改逻辑，明确检查 globals['config'] 中是否存在该键，而不是依赖 or 的短路求值。
| assert __key in self._supported_configs, f'Config {__key} is not supported' | |
| return globals['config'].get(__key) or config[__key] | |
| assert __key in self._supported_configs, f'Config {__key} is not supported' | |
| if __key in globals['config']: | |
| return globals['config'][__key] | |
| return config[__key] |
| def __init__(self, model: str = None, source: str = None, base_url: str = None, stream: bool = True, | ||
| return_trace: bool = False, skip_auth: bool = True, type: Optional[str] = None, | ||
| api_key: str = None, static_params: Optional[StaticParams] = None, **kwargs): | ||
| assert model is None, 'model should be given in forward method or global config.' | ||
| assert base_url is None, 'base_url should be given in forward method or global config.' | ||
| assert api_key is None, 'api_key should be given in forward method or global config.' | ||
| assert skip_auth is True, 'skip_auth should be True for dynamic LLM source.' | ||
| super().__init__(stream=stream, type=type, static_params=static_params) | ||
| self._return_trace = return_trace | ||
| self._kwargs = kwargs | ||
| self._suppliers: Dict[str, LLMBase] = {} | ||
|
|
||
| def _get_supplier(self): | ||
| if (source := globals.config['dynamic_llm_source']) is None: | ||
| raise KeyError('No source is configured for dynamic LLM source.') | ||
| if source not in self._suppliers: | ||
| self._suppliers[source] = getattr(lazyllm.online.chat, source)( | ||
| stream=self._stream, type=self._type, static_params=self._static_params, | ||
| skip_auth=True, return_trace=self._return_trace, **self._kwargs) | ||
| return self._suppliers[source] |
There was a problem hiding this comment.
动态 OnlineChatModule 的实现强制 skip_auth=True，这使得需要认证的动态数据源无法使用。
__init__方法中的assert skip_auth is True过于严格，限制了用户创建需要认证的动态模块。_get_supplier方法中硬编码了skip_auth=True来创建供应商实例。
建议修改此逻辑，允许为动态模块配置认证行为。可以将 skip_auth 参数存储在实例中，并在创建供应商时传递它。
| def __init__(self, model: str = None, source: str = None, base_url: str = None, stream: bool = True, | |
| return_trace: bool = False, skip_auth: bool = True, type: Optional[str] = None, | |
| api_key: str = None, static_params: Optional[StaticParams] = None, **kwargs): | |
| assert model is None, 'model should be given in forward method or global config.' | |
| assert base_url is None, 'base_url should be given in forward method or global config.' | |
| assert api_key is None, 'api_key should be given in forward method or global config.' | |
| assert skip_auth is True, 'skip_auth should be True for dynamic LLM source.' | |
| super().__init__(stream=stream, type=type, static_params=static_params) | |
| self._return_trace = return_trace | |
| self._kwargs = kwargs | |
| self._suppliers: Dict[str, LLMBase] = {} | |
| def _get_supplier(self): | |
| if (source := globals.config['dynamic_llm_source']) is None: | |
| raise KeyError('No source is configured for dynamic LLM source.') | |
| if source not in self._suppliers: | |
| self._suppliers[source] = getattr(lazyllm.online.chat, source)( | |
| stream=self._stream, type=self._type, static_params=self._static_params, | |
| skip_auth=True, return_trace=self._return_trace, **self._kwargs) | |
| return self._suppliers[source] | |
| def __init__(self, model: str = None, source: str = None, base_url: str = None, stream: bool = True, | |
| return_trace: bool = False, skip_auth: bool = False, type: Optional[str] = None, | |
| api_key: str = None, static_params: Optional[StaticParams] = None, **kwargs): | |
| assert model is None, 'model should be given in forward method or global config.' | |
| assert base_url is None, 'base_url should be given in forward method or global config.' | |
| assert api_key is None, 'api_key should be given in forward method or global config.' | |
| super().__init__(stream=stream, type=type, static_params=static_params) | |
| self._return_trace = return_trace | |
| self._skip_auth = skip_auth | |
| self._kwargs = kwargs | |
| self._suppliers: Dict[str, LLMBase] = {} | |
| def _get_supplier(self): | |
| if (source := globals.config['dynamic_llm_source']) is None: | |
| raise KeyError('No source is configured for dynamic LLM source.') | |
| if source not in self._suppliers: | |
| self._suppliers[source] = getattr(lazyllm.online.chat, source)( | |
| stream=self._stream, type=self._type, static_params=self._static_params, | |
| skip_auth=self._skip_auth, return_trace=self._return_trace, **self._kwargs) | |
| return self._suppliers[source] |
| paginator = self._s3_client.get_paginator('list_objects_v2') | ||
| to_delete: List[str] = [] | ||
| for page in paginator.paginate(Bucket=src_bucket, Prefix=src_prefix): | ||
| for obj in page.get('Contents', []): | ||
| rel = obj['Key'][len(src_prefix):] | ||
| self._s3_client.copy_object(CopySource={'Bucket': src_bucket, 'Key': obj['Key']}, | ||
| Bucket=dst_bucket, Key=dst_prefix + rel) | ||
| to_delete.append(obj['Key']) | ||
| for key in to_delete: | ||
| self._s3_client.delete_object(Bucket=src_bucket, Key=key) |
There was a problem hiding this comment.
在 move 方法中，对于目录的移动，当前实现是先复制所有对象，然后逐个删除源对象。当处理包含大量对象的目录时，逐个删除效率很低。
建议使用 S3 的 delete_objects API 进行批量删除，可以显著提升性能并减少 API 调用次数。可以在遍历每个分页（page）时，复制完该分页的对象后，就批量删除它们。
| paginator = self._s3_client.get_paginator('list_objects_v2') | |
| to_delete: List[str] = [] | |
| for page in paginator.paginate(Bucket=src_bucket, Prefix=src_prefix): | |
| for obj in page.get('Contents', []): | |
| rel = obj['Key'][len(src_prefix):] | |
| self._s3_client.copy_object(CopySource={'Bucket': src_bucket, 'Key': obj['Key']}, | |
| Bucket=dst_bucket, Key=dst_prefix + rel) | |
| to_delete.append(obj['Key']) | |
| for key in to_delete: | |
| self._s3_client.delete_object(Bucket=src_bucket, Key=key) | |
| paginator = self._s3_client.get_paginator('list_objects_v2') | |
| for page in paginator.paginate(Bucket=src_bucket, Prefix=src_prefix): | |
| contents = page.get('Contents', []) | |
| if not contents: | |
| continue | |
| for obj in contents: | |
| rel = obj['Key'][len(src_prefix):] | |
| self._s3_client.copy_object(CopySource={'Bucket': src_bucket, 'Key': obj['Key']}, | |
| Bucket=dst_bucket, Key=dst_prefix + rel) | |
| to_delete_payload = {'Objects': [{'Key': obj['Key']} for obj in contents]} | |
| self._s3_client.delete_objects(Bucket=src_bucket, Delete=to_delete_payload) |
1 participant
📌 PR 内容 / PR Description
本 PR 引入了两项相互配合的核心特性：会话级动态配置系统（
globals.config） 与OnlineChatModule动态 source 路由。1.
globals.config— 会话级动态配置新增
_GlobalConfig类（通过globals.config访问），在原有lazyllm.config（进程级静态配置）之上增加了一层会话（session）级动态覆盖能力：globals.config.add(name, type, default, env)— 注册一个支持动态覆盖的配置项，底层仍委托给lazyllm.config注册默认值和环境变量globals.config[key]— 读取时优先从当前 session 的globals['config']查找；若值为ConfigsDict（keyed by module id），则根据当前调用栈（globals.current_stack()）匹配最具体的配置，最终回退到lazyllm.configglobals.config[key] = value— 将值写入当前 session 的globals['config']，不影响其他 session与此同时，为支持按模块粒度的配置查找，在
Globals上新增了调用栈管理：push_stack/pop_stack/current_stack/stack_enter(module_id)— 在模块forward调用期间维护调用栈，使ConfigsDict能按 module id 精准匹配新增
SessionConfigableBasemixin，为需要参与动态配置的对象提供统一的id、name、group_id身份标识，以及identities属性供配置查找使用。2.
OnlineChatModule动态 source 路由新增
_DynamicSourceRouterMixin与dynamic_model_config_context，使OnlineChatModule（以及后续的OnlineEmbeddingModule、OnlineMultiModalModule）支持运行时动态切换 source / model / url / skip_auth：source='dynamic'— 构造时不绑定具体 supplier，改为在每次forward时从globals.config['dynamic_model_configs']读取当前生效的配置，并懒加载对应 supplier（线程安全缓存）dynamic_auth=True— 支持在forward的 kwargs 或globals.config中动态传入 api_key，适用于多租户场景dynamic_chat_config(modules, source, model, url, skip_auth)— 上下文管理器，在with块内为指定模块（或'default'）临时覆盖配置，退出后自动恢复快照，线程/session 安全🔍 相关 Issue / Related Issue
✅ 变更类型 / Type of Change
🧪 如何测试 / How Has This Been Tested?
OnlineChatModule(source='dynamic', dynamic_auth=True)dynamic_chat_config上下文管理器在不同 session 中分别设置不同的 source/model/api_key，验证各 session 互不干扰globals.config.add注册的配置项在无 session 覆盖时正确回退到lazyllm.config默认值ConfigsDict按 module id 精准匹配配置，default键作为兜底⚡ 更新后的用法示例 / Usage After Update
globals.config写入的是当前 session 的配置，不同 session（线程/协程）隔离，不会互相影响ConfigsDict的查找顺序为：当前调用栈各层 module id →'default'，因此模块级配置优先于全局默认dynamic_auth=True要求source='dynamic'且skip_auth=False；若同时设置skip_auth=True则需提供base_url(source, skip_auth)为 key 进行线程安全缓存，相同参数不会重复创建