[D1] Refactor Ghost Text preview to non-destructive (fix streaming overwrite/caret)

Author: inliver233

issues/2026-02-10_23-01-37-ui-autosave-streaming-v2.csv

@@ -0,0 +1,6 @@
+ID,Title,Description,Acceptance,Test_Method,Tools,Dev_Status,Review1_Status,Regression_Status,Files,Dependencies,Notes
+D1,Refactor Ghost Text preview to non-destructive (fix streaming overwrite/caret),"Replace default document-mutating Ghost Text preview with a non-destructive overlay (DeepIntegratedGhostText). Add editor paint overlay rendering and make accept/cancel stable. Ensure streaming chunk rendering never modifies document text or moves caret; clear preview on doc switch/save. Add pytest-qt coverage for overwrite/caret chaos regressions.","Streaming/AI preview does not change editor.toPlainText() until user accepts; caret stays stable during streaming; Tab accept inserts completion once at correct position; Esc cancels and stops updates; stale chunks ignored; tests pass.",".tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_streaming_ghost_no_overwrite.py",none,DONE,DONE,TODO,"src/gui/editor/text_editor.py | src/gui/editor/deep_integrated_ghost_text.py | src/gui/editor/optimal_ghost_text.py | src/gui/editor/smart_completion_manager.py | tests/test_streaming_ghost_no_overwrite.py",none,"user_report: streaming overwrites existing text + caret chaos | test:.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_streaming_ghost_no_overwrite.py PASS | done_at:2026-02-10"
+D2,Scratch autosave/restore + flush on close,"Persist unbound scratch editor content to a recovery file on autosave and on app close; restore it on startup when no project is restored. Also flush pending autosave for project-backed editors on app close so quick exits don’t lose content. Add tests + manual restart checklist.", "Typing in scratch tab persists after restart; closing app quickly still saves latest edits (scratch + project docs); tests pass (or limited validation recorded with risk).",".tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_scratch_autosave_restore.py",none,TODO,TODO,TODO,"src/gui/editor/editor_panel.py | src/gui/editor/text_editor.py | src/main.py | src/core/config.py | tests/test_scratch_autosave_restore.py",none,"manual_checklist: 1) Start app, type in scratch tab, close immediately. 2) Reopen; expect content restored. 3) Open project, edit scene, close immediately; reopen project; expect content present."
+D3,Bottom status bar separator/overlap cleanup,"Reduce stacked separator lines and clipping in EnhancedStatusBar. Use a single separator strategy (either borders or VLines) with consistent heights/margins and remove duplicate/overlapping lines. Update status bar layout test accordingly.","Bottom bar shows a single clean row; no stacked/thick separator lines; labels are not clipped; test passes.",".tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_status_bar_layout.py",none,TODO,TODO,TODO,"src/gui/status/status_bar.py | tests/test_status_bar_layout.py",none,"user_report: bottom bar lines stacked/overlapping"
+D4,"Max tokens UI supports >=1,000,000","Ensure AI配置中心的最大tokens输入允许很大的值（>=1,000,000）且不会出现输入首位数字就被限制/截断的问题；保存/加载配置保持一致。添加pytest-qt回归测试。","Max tokens can be set to 1,000,000+ and persists after reopening AI配置中心; regression test passes.",".tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_ai_config_max_tokens_range.py",none,TODO,TODO,TODO,"src/gui/ai/unified_ai_config_dialog.py | tests/test_ai_config_max_tokens_range.py",none,"user_report: max_tokens max=3999 and typing '4' clamps; verify and prevent regression"
+D5,Verify provider streaming specs + docs/tests,"Use official docs (OpenAI/Anthropic/Gemini) to verify streaming endpoints/headers/params and SSE parsing requirements; adjust provider strategies and AIClient stream parsing if needed; add a short doc summary and tests for OpenAI+Claude streaming fixtures.","Provider streaming endpoints/params match official docs as of 2026-02-10; OpenAI/Claude streaming extraction tests pass; doc added.",".tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_provider_streaming_openai_claude.py",manual,TODO,TODO,TODO,"src/core/ai_client.py | src/core/ai_providers/openai.py | src/core/ai_providers/claude.py | src/core/ai_providers/gemini.py | docs/ai/streaming_providers.md | tests/test_provider_streaming_openai_claude.py",none,"requires web research for up-to-date streaming specs; keep tests offline via fixtures"

plan/2026-02-10_23-01-37-ui-autosave-streaming-v2.md

@@ -0,0 +1,84 @@
+---
+mode: plan
+task: UI+Autosave+Streaming fixes v2
+created_at: "2026-02-10T23:07:08+08:00"
+complexity: complex
+---
+
+# Plan: UI + Autosave + Streaming fixes v2
+
+## Goal
+- Fix streaming completion so it never overwrites existing document text and caret stays stable; output renders in-order like typing.
+- Ensure blank/scratch editing is auto-saved and restored reliably (including on app close) so content isn鈥檛 lost.
+- Clean up bottom status bar visual clutter (no stacked separators/overlaps) and keep key info readable.
+- Confirm max_tokens UI supports large values (>= 1,000,000) and config persists correctly.
+- Verify provider streaming request/response formats for OpenAI/Anthropic/Gemini against latest docs and keep code + tests aligned.
+
+## Scope
+- In:
+  - Ghost text/inline completion rendering (`src/gui/editor/*ghost*`, `src/gui/editor/smart_completion_manager.py`)
+  - Streaming request/parse plumbing (`src/core/ai_client.py`, `src/core/ai_providers/*`, `src/core/ai_qt_client.py`, `src/gui/ai/enhanced_ai_manager.py`)
+  - Autosave/session restore for scratch + project docs (`src/gui/editor/editor_panel.py`, `src/gui/editor/text_editor.py`, `src/main.py`, `src/core/config*.py`)
+  - Status bar UI (`src/gui/status/status_bar.py` + related tests)
+  - AI config center UI token/stream toggles (`src/gui/ai/unified_ai_config_dialog.py`)
+- Out:
+  - Changing project DB schema
+  - Adding new providers beyond existing list (only ensure existing auto-adaptation is correct)
+
+## Assumptions / Dependencies
+- Use venv python: `.tmp\\ane0305-venv-311\\Scripts\\python.exe`
+- PyQt6 + pytest-qt available for tests.
+- Provider streaming docs may change; rely on official docs as of 2026-02-10.
+
+## Phases
+1. Create plan + Issue CSV contract; validate CSV.
+2. Streaming render safety: move ghost preview to non-destructive overlay; add tests guarding no overwrite/cursor chaos.
+3. Autosave reliability: flush on close; persist/restore scratch doc; add tests + manual checklist.
+4. Status bar cleanup: remove redundant separators and fix sizing; update layout test.
+5. Provider streaming verification: web research + adjust provider strategies/SSE parsing as needed; add/refresh fixtures.
+6. Regression: run `python -m compileall -q src` and targeted pytest suite; mark Regression DONE via meta commit.
+
+## Tests & Verification
+- Streaming safety -> `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_streaming_ghost_no_overwrite.py`
+- Scratch autosave -> `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_scratch_autosave_restore.py`
+- Status bar -> `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_status_bar_layout.py`
+- Config UI -> `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_ai_config_max_tokens_range.py`
+- Provider streaming -> `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q tests/test_provider_streaming_openai_claude.py`
+- Regression -> `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m compileall -q src` + `.tmp\\ane0305-venv-311\\Scripts\\python.exe -m pytest -q`
+
+## Issue CSV
+- Path: issues/2026-02-10_23-01-37-ui-autosave-streaming-v2.csv
+- Must share the same timestamp/slug as this plan.
+
+## Tools / MCP
+- `manual` for UI verification.
+- `none` for code work; web search (non-MCP) for provider docs.
+
+## Acceptance Checklist
+- [ ] Streaming completion never modifies document text until user accepts.
+- [ ] Streaming output appears in-order like typing; cancel stops updates; stale chunks ignored.
+- [ ] Closing the app does not lose recently typed content (project docs and scratch).
+- [ ] New/empty project starts with empty content (no ghost/saved garbage).
+- [ ] Bottom status bar has clean separators, no stacked lines or clipping.
+- [ ] Max tokens UI supports >= 1,000,000 and persists.
+- [ ] All issues Dev/Review1/Regression are DONE with tests evidence in Notes.
+
+## Risks / Blockers
+- PyQt paint overlay differences across platforms/themes; needs manual QA.
+- Streaming SSE formats may evolve; fixtures might need updates.
+
+## Rollback / Recovery
+- Keep old Optimal ghost system code; allow toggling default back if deep overlay causes regressions.
+- For autosave: keep scratch autosave file separate and non-destructive.
+
+## Checkpoints
+- Commit + push per issue.
+- Meta commit for Regression_Status after batch passes.
+
+## References
+- src/gui/editor/text_editor.py
+- src/gui/editor/optimal_ghost_text.py
+- src/gui/editor/deep_integrated_ghost_text.py
+- src/gui/editor/smart_completion_manager.py
+- src/core/ai_client.py
+- src/gui/status/status_bar.py

src/gui/editor/deep_integrated_ghost_text.py

@@ -515,9 +515,17 @@ def accept_ghost_text(self) -> bool:
         self.clear_ghost_text()
 
         # 插入实际文本
-        cursor = QTextCursor(self.document)
+        try:
+            cursor = self.text_editor.textCursor()
+        except Exception:
+            cursor = QTextCursor(self.document)
+
         cursor.setPosition(position)
         cursor.insertText(ghost_text)
+        try:
+            self.text_editor.setTextCursor(cursor)
+        except Exception:
+            pass
 
         # 发射信号
         self.ghost_text_accepted.emit(ghost_text)
@@ -833,4 +841,4 @@ def enhanced_key_press_event(event: QKeyEvent):
         logger.error(f"ghost_text_manager missing show_completion method! Type: {type(ghost_text_manager)}")
         raise AttributeError("DeepIntegratedGhostText instance missing show_completion method")
 
-    return ghost_text_manager
+    return ghost_text_manager

src/gui/editor/smart_completion_manager.py

@@ -234,6 +234,7 @@ def _bind_ghost_signals(self) -> None:
 
     def _on_ghost_completion_accepted(self, accepted_text: str) -> None:
         try:
+            self.stop_streaming_ai_completion()
             if self._ghost_state_manager:
                 self._ghost_state_manager.accept_with_text(accepted_text)
             self._completion_state_machine.set_state(CompletionState.APPLIED)
@@ -242,6 +243,7 @@ def _on_ghost_completion_accepted(self, accepted_text: str) -> None:
 
     def _on_ghost_completion_rejected(self) -> None:
         try:
+            self.stop_streaming_ai_completion()
             if self._ghost_state_manager:
                 self._ghost_state_manager.force_idle()
             self._completion_state_machine.set_state(CompletionState.CANCELLED)
@@ -722,6 +724,17 @@ def update_streaming_ai_completion(self, chunk_text: str, context: dict) -> None
             self._redetect_ghost_text_system()
         if not self._ghost_completion:
             return
+        # If the user has moved the caret since the request was started, stop rendering
+        # to avoid "jumping" previews and confusing cursor/overwrite behavior.
+        try:
+            anchor_pos = context.get("cursor_position") if isinstance(context, dict) else None
+            if isinstance(anchor_pos, int) and anchor_pos >= 0:
+                current_pos = self._text_editor.textCursor().position()
+                if current_pos != anchor_pos:
+                    self.stop_streaming_ai_completion()
+                    return
+        except Exception:
+            pass
         try:
             self._streaming_typed_controller.on_chunk(chunk_text, context or {})
         except Exception as exc:  # noqa: BLE001

src/gui/editor/text_editor.py

@@ -44,7 +44,7 @@
 
 logger = logging.getLogger(__name__)
 
-DEFAULT_GHOST_TEXT_SYSTEM = "optimal"  # "optimal" | "deep"
+DEFAULT_GHOST_TEXT_SYSTEM = "deep"  # "optimal" | "deep" (deep is non-destructive; safer for streaming/autosave)
 ALLOW_GHOST_TEXT_FALLBACK_TO_DEEP = True
 
 
@@ -245,10 +245,33 @@ def __init__(self, config: Config, shared: Shared, parent=None):
         logger.info("Intelligent text editor initialized")
 
     def paintEvent(self, event: QPaintEvent):
-        """简化的paintEvent - OptimalGhostText无需特殊渲染"""
-        # OptimalGhostText直接在文档中插入格式化文本，无需额外渲染
+        """Paint editor text, then (optionally) paint Ghost Text overlay.
+
+        DeepIntegratedGhostText is non-destructive and relies on an overlay render pass.
+        OptimalGhostText inserts formatted text into the document and does not require overlay rendering.
+        """
         super().paintEvent(event)
 
+        ghost = getattr(self, "_ghost_completion", None)
+        render_fn = getattr(ghost, "render_ghost_text", None)
+        if not callable(render_fn):
+            return
+
+        try:
+            painter = QPainter(self.viewport())
+            try:
+                painter.setClipRect(event.rect())
+            except Exception:
+                pass
+            render_fn(painter)
+        except Exception:
+            logger.debug("Ghost text overlay paint failed", exc_info=True)
+        finally:
+            try:
+                painter.end()
+            except Exception:
+                return
+
     # Legacy ghost text methods removed - replaced with OptimalGhostText
 
     def set_ghost_text(self, text: str, cursor_position: int):
@@ -877,6 +900,11 @@ def _detect_concepts(self):
 
     def set_document_content(self, content: str, document_id: str = None):
         """设置文档内容"""
+        # Ensure any in-flight Ghost Text preview does not bleed into the new document UI.
+        try:
+            self.clear_ghost_text()
+        except Exception:
+            pass
         self.setPlainText(content)
         self._current_document_id = document_id
         self._last_save_content = content
@@ -894,6 +922,12 @@ def is_modified(self) -> bool:
 
     def save_document(self) -> bool:
         """保存文档（优先持久化到项目数据库）。"""
+        # Never persist preview-only Ghost Text into the document storage.
+        try:
+            self.clear_ghost_text()
+        except Exception:
+            pass
+
         if not self._is_modified:
             return True
 

tests/test_streaming_ghost_no_overwrite.py

@@ -0,0 +1,84 @@
+from __future__ import annotations
+
+import os
+
+import pytest
+
+os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")
+
+pytest.importorskip("PyQt6")
+pytest.importorskip("pytestqt")
+
+from PyQt6.QtCore import Qt
+
+from core.config import Config
+from core.shared import Shared
+from gui.editor.text_editor import IntelligentTextEditor
+
+
+def test_streaming_ghost_preview_does_not_overwrite_document(qtbot) -> None:
+    config = Config()
+    shared = Shared(config)
+
+    editor = IntelligentTextEditor(config=config, shared=shared)
+    qtbot.addWidget(editor)
+    editor.show()
+    editor.setFocus()
+
+    editor.set_document_content("Hello ", document_id="doc1")
+    cursor = editor.textCursor()
+    cursor.movePosition(cursor.MoveOperation.End)
+    editor.setTextCursor(cursor)
+
+    anchor_pos = editor.textCursor().position()
+    original_text = editor.toPlainText()
+
+    context = {"request_id": "r1", "task_key": "t1", "cursor_position": anchor_pos}
+    editor._smart_completion.update_streaming_ai_completion("World", context)
+
+    ghost = getattr(editor, "_ghost_completion", None)
+    qtbot.waitUntil(lambda: getattr(ghost, "_current_ghost_text", "") == "World", timeout=1500)
+
+    assert editor.toPlainText() == original_text
+    assert editor.textCursor().position() == anchor_pos
+
+    # Accept preview with Tab.
+    qtbot.keyClick(editor, Qt.Key.Key_Tab)
+    qtbot.wait(50)
+
+    assert editor.toPlainText() == "Hello World"
+    assert editor.textCursor().position() == len("Hello World")
+
+
+def test_streaming_preview_stops_when_caret_moves(qtbot) -> None:
+    config = Config()
+    shared = Shared(config)
+
+    editor = IntelligentTextEditor(config=config, shared=shared)
+    qtbot.addWidget(editor)
+    editor.show()
+    editor.setFocus()
+
+    editor.set_document_content("ABC", document_id="doc1")
+    cursor = editor.textCursor()
+    cursor.movePosition(cursor.MoveOperation.End)
+    editor.setTextCursor(cursor)
+
+    anchor_pos = editor.textCursor().position()
+    context = {"request_id": "r1", "task_key": "t1", "cursor_position": anchor_pos}
+
+    editor._smart_completion.update_streaming_ai_completion("X", context)
+    qtbot.wait(50)
+
+    # Move caret away from the anchor position.
+    cursor = editor.textCursor()
+    cursor.setPosition(0)
+    editor.setTextCursor(cursor)
+
+    # Further chunks should stop streaming rendering instead of jumping to the new caret position.
+    editor._smart_completion.update_streaming_ai_completion("Y", context)
+    qtbot.wait(50)
+
+    ghost = getattr(editor, "_ghost_completion", None)
+    assert not bool(getattr(ghost, "_current_ghost_text", ""))
+    assert editor.toPlainText() == "ABC"