feat: API1.1 #434

CaralHsi · 2025-11-03T06:32:54Z

Description

Summary:

•	Memory & Retrieval Core: async add (plain & preference), Preference Memory pipeline, Reranker strategy suite, BM25 for TreeTextMemory, MemReader structural parsing.
•	Scheduler & Observability: API scheduler modularization (schema/utils/analyzer), Redis ORM for history sync & mixture search, metrics + request logs, Nacos-based dynamic config.
•	Data & Infra: PolarDB graph backend with pool/timeout & fixes, unified graph factory (Nebula/Neo4j/PolarDB), Milvus interface & item optimizations, enhanced logging.
•	Evaluation: PrefEval standardization; LoCoMo/LongMemEval/PersonaMem pipeline upgrades; new utilities (e.g., mirix_utils.py).
•	Stability & Fixes: query scheduling, message schema, Tree search inputs, self-input prompts, SQLite list users, PolarDB value mapping; pool/timeout tuning, usage data removal, graph-call toggle.
•	Compatibility: adopt new async path/schemas/metrics; align preference fields; migrate tests to Redis ORM; configure PolarDB pool/timeout before rollout.


•	记忆与检索内核：新增 async add（明文/偏好）、偏好记忆全链路、Reranker 策略集、TreeTextMemory 引入 BM25、MemReader 结构化解析。
•	调度与可观测性：API 调度模块化（schema/utils/analyzer）、Redis ORM（历史同步与混合搜索）、metrics 指标与请求日志、Nacos 动态配置。
•	数据与基础设施：PolarDB 图后端（连接池/超时与修复）、图工厂统一（Nebula/Neo4j/PolarDB）、Milvus 接口与数据项优化、日志链路增强。
•	评估体系：PrefEval 字段标准化；LoCoMo/LongMemEval/PersonaMem 评测升级；新增工具（如 mirix_utils.py）。
•	稳定性与修复：修复查询调度、消息 schema、树检索输入、自输入提示、SQLite 用户列表、PolarDB 值映射；连接池/超时优化、移除 usage 数据、图调用开关。
•	兼容性：适配异步路径/新 schema/metrics；偏好字段按新规范；测试迁移至 Redis ORM；启用 PolarDB 前配置连接池与超时并压测。

Fix: #424 #426 #423 #443 #384 #406 #445

•	Scheduler/Query: fixed query-schedule edge cases (#424).
•	Schemas: corrected message schema inconsistencies (#426).
•	Search I/O: fixed TreeTextMemory searcher input mismatch (#423).
•	Prompts: fixed self-input prompt error (#443).
•	Storage: fixed SQLite list-users error (#384).
•	Graph/DB: corrected PolarDB value/graph issues (#406, #445).

•	调度/查询：修复查询调度边界问题（#424）。
•	Schema：修正消息 schema 不一致（#426）。
•	检索 I/O：修复 Tree 搜索器输入不匹配（#423）。
•	提示：修复自输入提示错误（#443）。
•	存储：修复 SQLite 用户列表错误（#384）。
•	图/数据库：修正 PolarDB 值与图相关问题（#406、#445）。

Docs Issue/PR: (docs-issue-or-pr-link)

Reviewer: @(reviewer)

Checklist:

I have performed a self-review of my own code | 我已自行检查了自己的代码
I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR（如果适用）
I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR（如果适用）
I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

fix: sqlite users error

#373) * feat: define mem-read schedular message&consumer; add async mem-reader mode in core; * feat: add fast/fine mode in mem-reader; * feat: add mem-reader in scheduler * feat: change async remove * feat: modify async-add in core.py * feat: add 'remove and refresh memory in schedular' * feat: add naive fast mode in mem-reader * feat: finish fast mode in mem-reader * feat: add token-based window splitting and concurrency improvements * feat: add split chunker into mode in simple struct mem reader * feat: update async-mode add * chore: update gitignore * feat: improve database note write performance * feat: fix mem-read scheduler * fix: nebula group-by bug * fix: bug in adding mem scheduler * fix: nebula index; mem-reader chat-time; * format: searcher * fix: some bug in shceduler and mem-reader * feat: add mem-organize in scheduler * feat: add tree.mode to config; modify scheduler config * fix: test bug

* feat: check nodes existence * feat: use different template for different language input * feat: use different template for different language input * fix: eval script * feat: memos-api eval scripts * feat: mem reader * feat: 实现äºprefeval memos-api evaluation scripts * refactor:format code * feat: add PersonaMem eval scripts * docs(evaluation): update PersonaMem eval readme * feat:memos-api ingest batch message * feat: refactor search * feat: refactor search * update: add api for memory * feat: add memory api return memory and memory type * refactor(server):重构服务器路由模块以优化内存管理 * format: ruff format code * feat(server): 增加LLM最大令牌数 * test * fix: user query embedding for search * count memory_size by user * fix(server):修复记忆读取逻辑中的列表展开问题 * feat(nebular):优化图数据库查询性能 * refactor(memory): - 移除了对 `_refresh_memory_size` 方法的调用- 保留原有逻辑以便后续恢复或重构 * feat: remove user idx_memory_user_name * feat(graph):优化Nebula图数据库查询性能 * feat: rollback remove_oldest_memory * feat:nebula gql add index * feat: align code * feat: update memos_api * feat: update memos_api * feat: 更新默认选项 * feat:memory client * feat:refactor lme * feat: memu & supermemory client * feat: locomo memu * feat: locomo supermemory * New 'add' and 'process' modes. * feat: lme supermemory & memu * feat: default args * api and local * api and local * memobase fix * memos fix * default args * fix memos-api search data * prefeval pipeline * fix lme memos-api * personamem pipeline * personamem pipeline * lme scrips * align dev * format code * refactor: remove old files * format code * pm and prefeval pipeline * format code * format code * pm and prefeval pipeline * pm and prefeval pipeline * pm and prefeval pipeline * format code * format code * pref pipeline * add search response mode * add search response mode * update readme and example * update mem0 api * pm mem0 * fix MEMOBASE api * update pm and prefeval pipepline for frames * update pm and prefeval readme * format code * fix memobase api * fix memobase api * format code * format code * fix format * fix format * fix format --------- Co-authored-by: 2Rant <[email protected]> Co-authored-by: fridayL <[email protected]>

* debug an error function name * feat: Add DynamicCache compatibility for different transformers versions - Fix build_kv_cache method in hf.py to handle both old and new DynamicCache structures - Support new 'layers' attribute with key_cache/value_cache or keys/values - Maintain backward compatibility with direct key_cache/value_cache attributes - Add comprehensive error handling and logging for unsupported structures - Update move_dynamic_cache_htod function in kv.py for cross-version compatibility - Handle layers-based structure in newer transformers versions - Support alternative attribute names (keys/values vs key_cache/value_cache) - Preserve original functionality for older transformers versions - Add comprehensive tests for DynamicCache compatibility - Test activation memory update with mock DynamicCache layers - Verify layers attribute access across different transformers versions - Fix scheduler logger mock to include memory_manager attribute This resolves AttributeError issues when using different versions of the transformers library and ensures robust handling of DynamicCache objects. debug * feat: implement APIAnalyzerForScheduler for memory operations - Add APIAnalyzerForScheduler class with search/add operations - Support requests and http.client with connection reuse - Include comprehensive error handling and dynamic configuration - Add English test suite with real-world conversation scenarios * feat: Add search_ws API endpoint and enhance API analyzer functionality - Add search_ws endpoint in server_router.py for scheduler-enabled search - Fix missing imports: time module, SearchRequest class, and get_mos_product_instance function - Implement search_ws method in api_analyzer.py with HTTP client support - Add _search_ws_with_requests and _search_ws_with_http_client private methods - Include search_ws usage example in demonstration code - Enhance scheduler and dispatcher capabilities for improved memory management - Expand test coverage to ensure functionality stability This update primarily strengthens the memory scheduling system's search capabilities, providing users with more flexible API interface options. * fix: resolve test failures and warnings in test suite - Fix Pydantic serialization warning in test_memos_chen_tang_hello_world * Add warnings filter to suppress UserWarning from Pydantic serialization - Fix KeyError: 'past_key_values' in test_build_kv_cache_and_generation * Update mock configuration to properly return forward_output with past_key_values * Add DynamicCache version compatibility handling in test mocks * Support both old and new transformers versions with layers/key_cache attributes * Improve assertion logic to check all model calls for required parameters - Update base_scheduler.py to use centralized DEFAULT_MAX_INTERNAL_MESSAGE_QUEUE_SIZE constant * Add import for DEFAULT_MAX_INTERNAL_MESSAGE_QUEUE_SIZE from general_schemas * Replace hardcoded value 100 with configurable constant (1000) All tests now pass successfully with proper version compatibility handling. * feat: add a test_robustness execution to test thread pool execution * feat: optimize scheduler configuration and API search functionality - Add DEFAULT_TOP_K and DEFAULT_CONTEXT_WINDOW_SIZE global constants in general_schemas.py - Update base_scheduler.py to use global default values instead of hardcoded numbers - Fix SchedulerConfigFactory initialization issue by using keyword argument expansion - Resolve UnboundLocalError variable conflict in search_memories_ws function - Fix indentation and parameter issues in OptimizedScheduler search_for_api method - Improve code standardization and maintainability * feat: Add Redis auto-initialization with fallback strategies - Add auto_initialize_redis() with config/env/local fallback - Move Redis logic from dispatcher_monitor to redis_service - Update base_scheduler to use auto initialization - Add proper resource cleanup and error handling * feat: add database connection management to ORM module - Add MySQL engine loading from environment variables in BaseDBManager - Add Redis connection loading from environment variables in BaseDBManager - Enhance database configuration validation and error handling - Complete database adapter infrastructure for ORM module - Provide unified database connection management interface This update provides comprehensive database connection management capabilities for the mem_scheduler module, supporting dynamic MySQL and Redis configuration loading from environment variables, establishing reliable data persistence foundation for scheduling services and API services. * remove part of test * feat: add Redis-based ORM with multiprocess synchronization - Add RedisDBManager and RedisLockableORM classes - Implement atomic locking mechanism for concurrent access - Add merge functionality for different object types - Include comprehensive test suite and examples - Fix Redis key type conflicts in lock operations * fix: resolve scheduler module import and Redis integration issues * revise naive memcube creation in server router * remove long-time tests in test_scheduler * remove redis test which needs .env

* add preference text memory * finish milvus support * add new builder * finish prefer textual memory base level * modify code struct * modify pref module * implement remain preference function * modify preference.py * modify bug in milvus * finish debug * modify user pref user id code * modify bug in milvus * finish debug in core * repair bug in milvus get_all * add pref mem esarch time in core * modify search for pref mem in product..py * add simple pref memos example * modify bug in examples/mem_os/simple_prefs_memos_product.py * repair bug in user id related part * modify search * repair bug in slow update * modify define error in extractor -> extract_implicit_preferences * reapair define error in extractor and modify split func in spliter * modify code * modify adder * optimize the code * repair bug in adder and extractor * finish make test and make pre-commit * repair bug in preference * add memory field for milvusvecdbitem and modify related module * pref code clean * modify prompt of extractor * modify extractor * add reranker to pref mem * remove assember in pref mem * modify code * add op trace based update method in add * modify slow update in adder * modify implicit part code in extractor and add duplicate in utils * modify depulicate threshold * modify api config * reapir bug in adder about search relate * repair bug in core , dupicate search * add pref to new naive cube and server api * add async pref add by mem_schedular * modify * replace print to logger * repair bug from make pre-commit * inst cplt * align to liji cloud server * repair pkg problem * modify example of pref * pre_commit * fix api bug * merge inst_cplt to dev * fix pre commit * fix pre commit * fix pre commit error * modify code fllow reviewer * fix bug in make pre_commit * repair bug in server router * fix pre commit bug --------- Co-authored-by: yuan.wang <[email protected]>

* feat:change reranking source filed * fix: code ci * feat: add reranker strategy * fix: code suffix * fix: code suffix * fix:change strategy name * fix: code format * feat: update memory strategies * fix: code ci --------- Co-authored-by: CaralHsi <[email protected]>

* modify code in evaluation * modify code in evaluation --------- Co-authored-by: yuan.wang <[email protected]> Co-authored-by: chunyu li <[email protected]>

Co-authored-by: yuan.wang <[email protected]>

* add polardb.py * add polardb.py * add polar factory * delete * update get_memory_count * update get_memory_count * update node_not_exist * update remove_oldest_memory * fix * update get_node * update get_node * update update_node * update delete_node * add edge * add create_extension,create_graph,create_edge * add add_edge * edge_exist * edge_exist * edge_exist * update edge_exists * update polardb.py * update get_children_with_embeddings * update get_children_with_embeddings * update get_subgraph * update get_grouped_counts * update get_all_memory_items * update export_graph * remove * insert Memory * fix add_node * fix polardb.py * fix * fix get_subgraph * fix * get_grouped_counts * update get_by_metadata * get_grouped_counts * update get_grouped_counts * update get_grouped_counts * get_grouped_counts * update get_nodes * update search_by_embedding filter user_name * update search_by_embedding filter user_name * add filter user_name for update_node * get_structure_optimization_candidates * add filter user_name for update_node * fix * fix * fix * feat: 增加polardb的启动配置 * fix * fix * fix get_structure_optimization_candidates * fix get_all_memory_items * fix get_all_memory_items * remove embedding for get_nodes * fix get_structure_optimization_candidates * add _parse_node_new * update get_all_memory_items * update get_all_memory_items * update get_all_memory_items for include_embedding * feat: server router add polardb config * feat: server router add polardb config * update get_all_memory_items for include_embedding False * update get_all_memory_items for include_embedding False * fix * fix get_all_memory_items * update get_all_memory_items for include_embedding False * fix get_all_memory_items * update get_all_memory_items for include_embedding False * update get_grouped_counts * update get_grouped_counts * add_node and graph_id * fix * fix get_all_memory_items false * fix * fix get_all_memory_items true * fix * fix * fix * fix export_graph * fix export_graph * fix get_by_metadata * update get_neighbors_by_tag * update get_neighbors_by_tag * update get_neighbors_by_tag * fix * fix * add import_graph * fix * add get_edges * add clear * get_neighbors_by_tag * get_neighbors_by_tag * update get_by_metadata * search_by_emdedding remove embedding * fix:parseJson.py * fix:get_my_metadata * fix * fix get_by_metadata result * update polardb.py * fix _coerce_metadata * feat: add rerank time * feat: add rerank time * fix:node_not_exist * import node * import node * feat: fix merge_config_with_default * import node * fix * fix * feat: fix polardb * feat: fix scheduler method name * fix get_by_metadata for "query": "How long ago was Caroline's 18th birthday?" * fix get_by_metadata for "query": "How long ago was Caroline's 18th birthday?" * fix get_node format_param_value * feat: fix CONFIG * fix * feat: fix import * feat: delete test file * feat: fix polardb * feat: fix recall * Comment out unused configuration handling code Commented out code related to auto_create and embedding_dimension handling. * fix * feat: fix polardb * import polardb * feat: fix polardb * fix * feat: fix polardb * fix * fix * feat: fix polardb * feat: delete polardb * feat: fix utils * feat: fix polardb * feat: format polardb * feat: format utils --------- Co-authored-by: ccl <[email protected]> Co-authored-by: liji <[email protected]> Co-authored-by: CaralHsi <[email protected]>

…401) * feat: add content and remove log waring * fix: wrong format * fix: fix code format

…398) * debug an error function name * feat: Add DynamicCache compatibility for different transformers versions - Fix build_kv_cache method in hf.py to handle both old and new DynamicCache structures - Support new 'layers' attribute with key_cache/value_cache or keys/values - Maintain backward compatibility with direct key_cache/value_cache attributes - Add comprehensive error handling and logging for unsupported structures - Update move_dynamic_cache_htod function in kv.py for cross-version compatibility - Handle layers-based structure in newer transformers versions - Support alternative attribute names (keys/values vs key_cache/value_cache) - Preserve original functionality for older transformers versions - Add comprehensive tests for DynamicCache compatibility - Test activation memory update with mock DynamicCache layers - Verify layers attribute access across different transformers versions - Fix scheduler logger mock to include memory_manager attribute This resolves AttributeError issues when using different versions of the transformers library and ensures robust handling of DynamicCache objects. debug * feat: implement APIAnalyzerForScheduler for memory operations - Add APIAnalyzerForScheduler class with search/add operations - Support requests and http.client with connection reuse - Include comprehensive error handling and dynamic configuration - Add English test suite with real-world conversation scenarios * feat: Add search_ws API endpoint and enhance API analyzer functionality - Add search_ws endpoint in server_router.py for scheduler-enabled search - Fix missing imports: time module, SearchRequest class, and get_mos_product_instance function - Implement search_ws method in api_analyzer.py with HTTP client support - Add _search_ws_with_requests and _search_ws_with_http_client private methods - Include search_ws usage example in demonstration code - Enhance scheduler and dispatcher capabilities for improved memory management - Expand test coverage to ensure functionality stability This update primarily strengthens the memory scheduling system's search capabilities, providing users with more flexible API interface options. * fix: resolve test failures and warnings in test suite - Fix Pydantic serialization warning in test_memos_chen_tang_hello_world * Add warnings filter to suppress UserWarning from Pydantic serialization - Fix KeyError: 'past_key_values' in test_build_kv_cache_and_generation * Update mock configuration to properly return forward_output with past_key_values * Add DynamicCache version compatibility handling in test mocks * Support both old and new transformers versions with layers/key_cache attributes * Improve assertion logic to check all model calls for required parameters - Update base_scheduler.py to use centralized DEFAULT_MAX_INTERNAL_MESSAGE_QUEUE_SIZE constant * Add import for DEFAULT_MAX_INTERNAL_MESSAGE_QUEUE_SIZE from general_schemas * Replace hardcoded value 100 with configurable constant (1000) All tests now pass successfully with proper version compatibility handling. * feat: add a test_robustness execution to test thread pool execution * feat: optimize scheduler configuration and API search functionality - Add DEFAULT_TOP_K and DEFAULT_CONTEXT_WINDOW_SIZE global constants in general_schemas.py - Update base_scheduler.py to use global default values instead of hardcoded numbers - Fix SchedulerConfigFactory initialization issue by using keyword argument expansion - Resolve UnboundLocalError variable conflict in search_memories_ws function - Fix indentation and parameter issues in OptimizedScheduler search_for_api method - Improve code standardization and maintainability * feat: Add Redis auto-initialization with fallback strategies - Add auto_initialize_redis() with config/env/local fallback - Move Redis logic from dispatcher_monitor to redis_service - Update base_scheduler to use auto initialization - Add proper resource cleanup and error handling * feat: add database connection management to ORM module - Add MySQL engine loading from environment variables in BaseDBManager - Add Redis connection loading from environment variables in BaseDBManager - Enhance database configuration validation and error handling - Complete database adapter infrastructure for ORM module - Provide unified database connection management interface This update provides comprehensive database connection management capabilities for the mem_scheduler module, supporting dynamic MySQL and Redis configuration loading from environment variables, establishing reliable data persistence foundation for scheduling services and API services. * remove part of test * feat: add Redis-based ORM with multiprocess synchronization - Add RedisDBManager and RedisLockableORM classes - Implement atomic locking mechanism for concurrent access - Add merge functionality for different object types - Include comprehensive test suite and examples - Fix Redis key type conflicts in lock operations * fix: resolve scheduler module import and Redis integration issues * revise naive memcube creation in server router * remove long-time tests in test_scheduler * remove redis test which needs .env * refactor all codes about mixture search with scheduler * fix: resolve Redis API synchronization issues and implement search API with reranker - Fix running_entries to running_task_ids migration across codebase - Update sync_search_data method to properly handle TaskRunningStatus - Correct variable naming and logic in API synchronization flow - Implement search API endpoint with reranker functionality - Update test files to reflect new running_task_ids convention - Ensure proper Redis state management for concurrent tasks * remove a test for api module * revise to pass the test suite * address some bugs to make mix_search normally running * modify codes according to evaluation logs --------- Co-authored-by: CaralHsi <[email protected]>

* feat: check nodes existence * feat: use different template for different language input * feat: use different template for different language input * fix: eval script * feat: memos-api eval scripts * feat: mem reader * feat: 实现äºprefeval memos-api evaluation scripts * refactor:format code * feat: add PersonaMem eval scripts * docs(evaluation): update PersonaMem eval readme * feat:memos-api ingest batch message * feat: refactor search * feat: refactor search * update: add api for memory * feat: add memory api return memory and memory type * refactor(server):重构服务器路由模块以优化内存管理 * format: ruff format code * feat(server): 增加LLM最大令牌数 * test * fix: user query embedding for search * count memory_size by user * fix(server):修复记忆读取逻辑中的列表展开问题 * feat(nebular):优化图数据库查询性能 * refactor(memory): - 移除了对 `_refresh_memory_size` 方法的调用- 保留原有逻辑以便后续恢复或重构 * feat: remove user idx_memory_user_name * feat(graph):优化Nebula图数据库查询性能 * feat: rollback remove_oldest_memory * feat:nebula gql add index * feat: align code * feat: update memos_api * feat: update memos_api * feat: 更新默认选项 * feat:memory client * feat:refactor lme * feat: memu & supermemory client * feat: locomo memu * feat: locomo supermemory * New 'add' and 'process' modes. * feat: lme supermemory & memu * feat: default args * api and local * api and local * memobase fix * memos fix * default args * fix memos-api search data * prefeval pipeline * fix lme memos-api * personamem pipeline * personamem pipeline * lme scrips * align dev * format code * refactor: remove old files * format code * pm and prefeval pipeline * format code * format code * pm and prefeval pipeline * pm and prefeval pipeline * pm and prefeval pipeline * format code * format code * pref pipeline * add search response mode * add search response mode * update readme and example * update mem0 api * pm mem0 * fix MEMOBASE api * update pm and prefeval pipepline for frames * update pm and prefeval readme * format code * fix memobase api * fix memobase api * format code * format code * fix format * fix format * fix format * mem0 api * memos batch add * add memos-api-online * add memos-api-online update readme * rollback manager * memos online api pref mem --------- Co-authored-by: 2Rant <[email protected]> Co-authored-by: fridayL <[email protected]> Co-authored-by: CaralHsi <[email protected]>

* feat: fix sources * feat: fix sources --------- Co-authored-by: chunyu li <[email protected]>

* feat: fix sources * feat: fix sources * feat: fix nebular * feat: fix polardb edges * feat: format polardb --------- Co-authored-by: chunyu li <[email protected]>

* feat: update log context * feat: update log context * feat: update mcp * feat: update mcp * feat: add error log * feat: add error log * feat: add error log * feat: update log * feat: add chat_time * feat: add chat_time * feat: add chat_time * feat: update log * feat: update log * feat: update log * feat: update log * feat: update log * feat: add arms * fix: format * fix: format * feat: add dockerfile * feat: add dockerfile * feat: add arms config * feat: update log * feat: add sleep time * feat: add sleep time * feat: update log * feat: delete dockerfile * feat: delete dockerfile * feat: update dockerfile * fix: conflict * feat: replace ThreadPool to context * feat: add timed log --------- Co-authored-by: chunyu li <[email protected]> Co-authored-by: harvey_xiang <[email protected]>

hotfix: memos playground

* add hybrid search and fine extractor * add dialog and modify spliter chunk * optmize the update and retriever code * modify pref field * add pref mem update srategy * add pref mem update srategy * fix bug in pre_commit --------- Co-authored-by: yuan.wang <[email protected]>

* feat: update manager for async add * feat: modify tree and simple_tree, TODO: STILL NOT ALIGN IN SOME FUNCTIONS * feat: modify schedule: add optional user_name in schedule message; modify user-name related graph query in scheduler * feat: finishe server router for async mode * feat: update graph db * fix: add label in core * feat: add tree mode in config * feat: default llm token 8000 * fix: thread * feat: search mode in client: fast * tests: fix

* fix * fix：nacos * feat: fix config Exception * feat: format config * feat: format config --------- Co-authored-by: ccl <[email protected]>

* feat: update manager for async add * feat: modify tree and simple_tree, TODO: STILL NOT ALIGN IN SOME FUNCTIONS * feat: modify schedule: add optional user_name in schedule message; modify user-name related graph query in scheduler * feat: finishe server router for async mode * feat: update graph db * fix: add label in core * feat: add tree mode in config * feat: default llm token 8000 * fix: thread * feat: search mode in client: fast * tests: fix * fix: add some log for memory_size in manager --------- Co-authored-by: chunyu li <[email protected]>

…king to 1 stage (#405) * debug an error function name * feat: Add DynamicCache compatibility for different transformers versions - Fix build_kv_cache method in hf.py to handle both old and new DynamicCache structures - Support new 'layers' attribute with key_cache/value_cache or keys/values - Maintain backward compatibility with direct key_cache/value_cache attributes - Add comprehensive error handling and logging for unsupported structures - Update move_dynamic_cache_htod function in kv.py for cross-version compatibility - Handle layers-based structure in newer transformers versions - Support alternative attribute names (keys/values vs key_cache/value_cache) - Preserve original functionality for older transformers versions - Add comprehensive tests for DynamicCache compatibility - Test activation memory update with mock DynamicCache layers - Verify layers attribute access across different transformers versions - Fix scheduler logger mock to include memory_manager attribute This resolves AttributeError issues when using different versions of the transformers library and ensures robust handling of DynamicCache objects. debug * feat: implement APIAnalyzerForScheduler for memory operations - Add APIAnalyzerForScheduler class with search/add operations - Support requests and http.client with connection reuse - Include comprehensive error handling and dynamic configuration - Add English test suite with real-world conversation scenarios * feat: Add search_ws API endpoint and enhance API analyzer functionality - Add search_ws endpoint in server_router.py for scheduler-enabled search - Fix missing imports: time module, SearchRequest class, and get_mos_product_instance function - Implement search_ws method in api_analyzer.py with HTTP client support - Add _search_ws_with_requests and _search_ws_with_http_client private methods - Include search_ws usage example in demonstration code - Enhance scheduler and dispatcher capabilities for improved memory management - Expand test coverage to ensure functionality stability This update primarily strengthens the memory scheduling system's search capabilities, providing users with more flexible API interface options. * fix: resolve test failures and warnings in test suite - Fix Pydantic serialization warning in test_memos_chen_tang_hello_world * Add warnings filter to suppress UserWarning from Pydantic serialization - Fix KeyError: 'past_key_values' in test_build_kv_cache_and_generation * Update mock configuration to properly return forward_output with past_key_values * Add DynamicCache version compatibility handling in test mocks * Support both old and new transformers versions with layers/key_cache attributes * Improve assertion logic to check all model calls for required parameters - Update base_scheduler.py to use centralized DEFAULT_MAX_INTERNAL_MESSAGE_QUEUE_SIZE constant * Add import for DEFAULT_MAX_INTERNAL_MESSAGE_QUEUE_SIZE from general_schemas * Replace hardcoded value 100 with configurable constant (1000) All tests now pass successfully with proper version compatibility handling. * feat: add a test_robustness execution to test thread pool execution * feat: optimize scheduler configuration and API search functionality - Add DEFAULT_TOP_K and DEFAULT_CONTEXT_WINDOW_SIZE global constants in general_schemas.py - Update base_scheduler.py to use global default values instead of hardcoded numbers - Fix SchedulerConfigFactory initialization issue by using keyword argument expansion - Resolve UnboundLocalError variable conflict in search_memories_ws function - Fix indentation and parameter issues in OptimizedScheduler search_for_api method - Improve code standardization and maintainability * feat: Add Redis auto-initialization with fallback strategies - Add auto_initialize_redis() with config/env/local fallback - Move Redis logic from dispatcher_monitor to redis_service - Update base_scheduler to use auto initialization - Add proper resource cleanup and error handling * feat: add database connection management to ORM module - Add MySQL engine loading from environment variables in BaseDBManager - Add Redis connection loading from environment variables in BaseDBManager - Enhance database configuration validation and error handling - Complete database adapter infrastructure for ORM module - Provide unified database connection management interface This update provides comprehensive database connection management capabilities for the mem_scheduler module, supporting dynamic MySQL and Redis configuration loading from environment variables, establishing reliable data persistence foundation for scheduling services and API services. * remove part of test * feat: add Redis-based ORM with multiprocess synchronization - Add RedisDBManager and RedisLockableORM classes - Implement atomic locking mechanism for concurrent access - Add merge functionality for different object types - Include comprehensive test suite and examples - Fix Redis key type conflicts in lock operations * fix: resolve scheduler module import and Redis integration issues * revise naive memcube creation in server router * remove long-time tests in test_scheduler * remove redis test which needs .env * refactor all codes about mixture search with scheduler * fix: resolve Redis API synchronization issues and implement search API with reranker - Fix running_entries to running_task_ids migration across codebase - Update sync_search_data method to properly handle TaskRunningStatus - Correct variable naming and logic in API synchronization flow - Implement search API endpoint with reranker functionality - Update test files to reflect new running_task_ids convention - Ensure proper Redis state management for concurrent tasks * remove a test for api module * revise to pass the test suite * address some bugs to make mix_search normally running * modify codes according to evaluation logs * feat: Optimize mixture search and enhance API client * feat: Add conversation_turn tracking for session-based memory search - Add conversation_turn field to APIMemoryHistoryEntryItem schema with default value 0 - Implement session counter in OptimizedScheduler to track turn count per session_id - Update sync_search_data method to accept and store conversation_turn parameter - Maintain session history with LRU eviction (max 5 sessions) - Rename conversation_id to session_id for consistency with request object - Enable direct access to session_id from search requests This feature allows tracking conversation turns within the same session, providing better context for memory retrieval and search history management. * adress time bug in monitor * revise simple tree * add mode to evaluation client; rewrite print to logger.info in db files --------- Co-authored-by: CaralHsi <[email protected]> Co-authored-by: chunyu li <[email protected]>

* update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]>

Revert "Feat: add recall strategy (#414)" This reverts commit a375911. Co-authored-by: chunyu li <[email protected]>

* update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]>

feat: remove usage data

* feat: change MOS_SCHEDULER_THREAD_POOL_MAX_WORKERS to 10000 * feat: add user_name to schedule server router * feat: roll back to old mem-reader-prompt * feat: add moniter in schedule * feat: set default MEMRADER_MAX_TOKENS to 8000

* update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test * turn off graph recall * turn off graph recall * turn off graph recall --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]> Co-authored-by: chunyu li <[email protected]>

* feat: check nodes existence * feat: use different template for different language input * feat: use different template for different language input * fix: eval script * feat: memos-api eval scripts * feat: mem reader * feat: 实现äºprefeval memos-api evaluation scripts * refactor:format code * feat: add PersonaMem eval scripts * docs(evaluation): update PersonaMem eval readme * feat:memos-api ingest batch message * feat: refactor search * feat: refactor search * update: add api for memory * feat: add memory api return memory and memory type * refactor(server):重构服务器路由模块以优化内存管理 * format: ruff format code * feat(server): 增加LLM最大令牌数 * test * fix: user query embedding for search * count memory_size by user * fix(server):修复记忆读取逻辑中的列表展开问题 * feat(nebular):优化图数据库查询性能 * refactor(memory): - 移除了对 `_refresh_memory_size` 方法的调用- 保留原有逻辑以便后续恢复或重构 * feat: remove user idx_memory_user_name * feat(graph):优化Nebula图数据库查询性能 * feat: rollback remove_oldest_memory * feat:nebula gql add index * feat: align code * feat: update memos_api * feat: update memos_api * feat: 更新默认选项 * feat:memory client * feat:refactor lme * feat: memu & supermemory client * feat: locomo memu * feat: locomo supermemory * New 'add' and 'process' modes. * feat: lme supermemory & memu * feat: default args * api and local * api and local * memobase fix * memos fix * default args * fix memos-api search data * prefeval pipeline * fix lme memos-api * personamem pipeline * personamem pipeline * lme scrips * align dev * format code * refactor: remove old files * format code * pm and prefeval pipeline * format code * format code * pm and prefeval pipeline * pm and prefeval pipeline * pm and prefeval pipeline * format code * format code * pref pipeline * add search response mode * add search response mode * update readme and example * update mem0 api * pm mem0 * fix MEMOBASE api * update pm and prefeval pipepline for frames * update pm and prefeval readme * format code * fix memobase api * fix memobase api * format code * format code * fix format * fix format * fix format * mem0 api * memos batch add * add memos-api-online * add memos-api-online update readme * rollback manager * memos online api pref mem * readme * update pref eval and pm scripts * add breakpoint in eval scripts * modify default param --------- Co-authored-by: 2Rant <[email protected]> Co-authored-by: fridayL <[email protected]> Co-authored-by: CaralHsi <[email protected]>

* add hybrid search and fine extractor * add dialog and modify spliter chunk * optmize the update and retriever code * modify pref field * add pref mem update srategy * add pref mem update srategy * fix bug in pre_commit * modify pref filed * fix bug * fix pre_commit * fix bug in adder * fast --------- Co-authored-by: yuan.wang <[email protected]>

* add hybrid search and fine extractor * add dialog and modify spliter chunk * optmize the update and retriever code * modify pref field * add pref mem update srategy * add pref mem update srategy * fix bug in pre_commit * modify pref filed * fix bug * fix pre_commit * fix bug in adder * fast * modify pref and adder mode * modify code * make pre_commit --------- Co-authored-by: yuan.wang <[email protected]>

* add hybrid search and fine extractor * add dialog and modify spliter chunk * optmize the update and retriever code * modify pref field * add pref mem update srategy * add pref mem update srategy * fix bug in pre_commit * modify pref filed * fix bug * fix pre_commit * fix bug in adder * fast * modify pref and adder mode * modify code * make pre_commit * fix pref_string for memos online api * modify code * modify code * modify code * pre comimt * modify code --------- Co-authored-by: yuan.wang <[email protected]>

* feat: update log context * feat: update log context * feat: update mcp * feat: update mcp * feat: add error log * feat: add error log * feat: add error log * feat: update log * feat: add chat_time * feat: add chat_time * feat: add chat_time * feat: update log * feat: update log * feat: update log * feat: update log * feat: update log * feat: add arms * fix: format * fix: format * feat: add dockerfile * feat: add dockerfile * feat: add arms config * feat: update log * feat: add sleep time * feat: add sleep time * feat: update log * feat: delete dockerfile * feat: delete dockerfile * feat: update dockerfile * fix: conflict * feat: replace ThreadPool to context * feat: add timed log * feat: add request log * feat: add request log --------- Co-authored-by: harvey_xiang <[email protected]>

* standardized preference field * fix pre_commit --------- Co-authored-by: yuan.wang <[email protected]> Co-authored-by: chunyu li <[email protected]>

* fix：parse_node * fix： * fix：conn --------- Co-authored-by: ccl <[email protected]> Co-authored-by: CaralHsi <[email protected]>

* feat: update log context * feat: update log context * feat: update mcp * feat: update mcp * feat: add error log * feat: add error log * feat: add error log * feat: update log * feat: add chat_time * feat: add chat_time * feat: add chat_time * feat: update log * feat: update log * feat: update log * feat: update log * feat: update log * feat: add arms * fix: format * fix: format * feat: add dockerfile * feat: add dockerfile * feat: add arms config * feat: update log * feat: add sleep time * feat: add sleep time * feat: update log * feat: delete dockerfile * feat: delete dockerfile * feat: update dockerfile * fix: conflict * feat: replace ThreadPool to context * feat: add timed log * feat: add request log * feat: add request log * feat: add source in request * feat: source --------- Co-authored-by: harvey_xiang <[email protected]> Co-authored-by: CaralHsi <[email protected]>

* feat: update log context * feat: update log context * feat: update mcp * feat: update mcp * feat: add error log * feat: add error log * feat: add error log * feat: update log * feat: add chat_time * feat: add chat_time * feat: add chat_time * feat: update log * feat: update log * feat: update log * feat: update log * feat: update log * feat: add arms * fix: format * fix: format * feat: add dockerfile * feat: add dockerfile * feat: add arms config * feat: update log * feat: add sleep time * feat: add sleep time * feat: update log * feat: delete dockerfile * feat: delete dockerfile * feat: update dockerfile * fix: conflict * feat: replace ThreadPool to context * feat: add timed log * feat: add request log * feat: add request log * feat: add source in request * feat: source * feat: revert context --------- Co-authored-by: harvey_xiang <[email protected]>

Co-authored-by: yuan.wang <[email protected]>

…st reader strategy template. (#448) * update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test * turn off graph recall * turn off graph recall * turn off graph recall * fix Searcher input bug * fix Searcher * fix Search * fix bug * adjust strategy reader * adjust strategy reader * adjust search config input * reformat code --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]> Co-authored-by: chunyu li <[email protected]>

* add breakpoint in eval scripts * feat(locomo): 支持断点续传 * format code * doc: Update readme eval result

* fix： * fix： * format * fix：id --------- Co-authored-by: ccl <[email protected]>

Revert "fix prompt error (#447)" This reverts commit 4c8f89a.

…et; adjust reader strategy template." (#453) Revert "fix: fix search config input bug; patch retrieve_utils path set; adju…" This reverts commit 88699f9.

Revert "Useless quotes (#450)" This reverts commit e62f72d.

Co-authored-by: yuan.wang <[email protected]> Co-authored-by: CaralHsi <[email protected]>

modify prompt Co-authored-by: yuan.wang <[email protected]>

* update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test * turn off graph recall * turn off graph recall * turn off graph recall * fix Searcher input bug * fix Searcher * fix Search * fix bug * adjust strategy reader * adjust strategy reader * adjust search config input * reformat code * re pr * format repair --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]> Co-authored-by: chunyu li <[email protected]>

* fix：parse_node * fix： * fix：conn * fix：getnode * fix： --------- Co-authored-by: ccl <[email protected]> Co-authored-by: CaralHsi <[email protected]>

* modify bug * modify bug * remove print --------- Co-authored-by: yuan.wang <[email protected]>

* fix: response error * fix: response error * fix: response error --------- Co-authored-by: harvey_xiang <[email protected]>

* doc: Update readme * feat(env): update .env.example * refactor(client): pref mem * fix pref type * chore(env): 更新环境变量配置文件

* fix: response error * fix: response error * fix: response error * feat: replace context thread --------- Co-authored-by: harvey_xiang <[email protected]> Co-authored-by: CaralHsi <[email protected]>

* fix:get_grouped_counts * fix: code format

* feat: tmp-close usage update, maybe reopen it later * test: withdraw update usage test

fridayL and others added 30 commits October 23, 2025 14:05

fix: sqlite list users error (#384)

d9f863e

fix: sqlite users error

modify code in evaluation (#392)

e069928

* modify code in evaluation * modify code in evaluation --------- Co-authored-by: yuan.wang <[email protected]> Co-authored-by: chunyu li <[email protected]>

fix bug in pref_mem return (#399)

84adda6

Co-authored-by: yuan.wang <[email protected]>

feat: fix mode (#400)

83a7c34

Feat: remove long waring for internet and add content for memreader (#…

018d759

…401) * feat: add content and remove log waring * fix: wrong format * fix: fix code format

feat: fix sources (#404)

1f6757d

* feat: fix sources * feat: fix sources --------- Co-authored-by: chunyu li <[email protected]>

fix porlar (#406)

e21f5bb

* feat: fix sources * feat: fix sources * feat: fix nebular * feat: fix polardb edges * feat: format polardb --------- Co-authored-by: chunyu li <[email protected]>

Hotfix: memos playground prompt reverse (#408)

f8859f1

hotfix: memos playground

feat: fix polardb graph (#411)

4ed7574

use nacos (#407)

6e219c4

* fix * fix：nacos * feat: fix config Exception * feat: format config * feat: format config --------- Co-authored-by: ccl <[email protected]>

Feat: add recall strategy (#414)

a375911

* update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]>

Revert "Feat: add recall strategy " (#415)

5b8893e

Revert "Feat: add recall strategy (#414)" This reverts commit a375911. Co-authored-by: chunyu li <[email protected]>

Feat： add new recall and verify (#416)

445c597

* update reader and search strategy * set strategy reader and search config * fix install problem * fix * fix test --------- Co-authored-by: 黑布林 <[email protected]> Co-authored-by: CaralHsi <[email protected]>

Feat: remove usage data (#417)

0765e1c

feat: remove usage data

feat: add moniter schedule (#419)

39a4f29

* feat: change MOS_SCHEDULER_THREAD_POOL_MAX_WORKERS to 10000 * feat: add user_name to schedule server router * feat: roll back to old mem-reader-prompt * feat: add moniter in schedule * feat: set default MEMRADER_MAX_TOKENS to 8000

Wang-Daoji and others added 28 commits October 30, 2025 21:35

Merge branch 'main' into dev

4f96241

Feat/standardized preference field (#440)

b3b0baa

* standardized preference field * fix pre_commit --------- Co-authored-by: yuan.wang <[email protected]> Co-authored-by: chunyu li <[email protected]>

update polardb pool timeout (#441)

cef9369

* fix：parse_node * fix： * fix：conn --------- Co-authored-by: ccl <[email protected]> Co-authored-by: CaralHsi <[email protected]>

feat: fix self-input prompt error (#443)

fd56f64

feat: fix polardb value (#445)

e79a9ab

fix prompt error (#447)

4c8f89a

Co-authored-by: yuan.wang <[email protected]>

eval result (#428)

fafc747

* add breakpoint in eval scripts * feat(locomo): 支持断点续传 * format code * doc: Update readme eval result

Useless quotes (#450)

e62f72d

* fix： * fix： * format * fix：id --------- Co-authored-by: ccl <[email protected]>

Revert "fix prompt error" (#452)

f3e7338

Revert "fix prompt error (#447)" This reverts commit 4c8f89a.

Revert "fix: fix search config input bug; patch retrieve_utils path s…

b8cd27b

…et; adjust reader strategy template." (#453) Revert "fix: fix search config input bug; patch retrieve_utils path set; adju…" This reverts commit 88699f9.

Revert "Useless quotes" (#454)

e07a1b4

Revert "Useless quotes (#450)" This reverts commit e62f72d.

fix prompt error (#455)

f67ca36

Co-authored-by: yuan.wang <[email protected]> Co-authored-by: CaralHsi <[email protected]>

Feat/remove pref rank prompt (#456)

7c4a74c

modify prompt Co-authored-by: yuan.wang <[email protected]>

Dev ccl1103 (#449)

65a2daf

* fix：parse_node * fix： * fix：conn * fix：getnode * fix： --------- Co-authored-by: ccl <[email protected]> Co-authored-by: CaralHsi <[email protected]>

Feat/fix bug 1031 (#459)

b37939c

* modify bug * modify bug * remove print --------- Co-authored-by: yuan.wang <[email protected]>

Fix/no response (#463)

ccbffae

* fix: response error * fix: response error * fix: response error --------- Co-authored-by: harvey_xiang <[email protected]>

doc: Update readme (#458)

4e500a9

* doc: Update readme * feat(env): update .env.example * refactor(client): pref mem * fix pref type * chore(env): 更新环境变量配置文件

Fix/no response (#464)

5e4f695

* fix: response error * fix: response error * fix: response error * feat: replace context thread --------- Co-authored-by: harvey_xiang <[email protected]> Co-authored-by: CaralHsi <[email protected]>

Fix: fix pg query for group error string (#465)

f640855

* fix:get_grouped_counts * fix: code format

feat: freeze usage update in Searcher (#466)

119bbe2

* feat: tmp-close usage update, maybe reopen it later * test: withdraw update usage test

CaralHsi marked this pull request as ready for review November 6, 2025 13:27

CaralHsi merged commit cc7122f into main Nov 6, 2025
40 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: API1.1 #434

feat: API1.1 #434

CaralHsi commented Nov 3, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

feat: API1.1 #434

feat: API1.1 #434

Conversation

CaralHsi commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist:

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

CaralHsi commented Nov 3, 2025 •

edited

Loading