forked from cline/cline
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
feat: Summarize task titles for legibility and scannability (WIP) #8640
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
brunobergher
wants to merge
1
commit into
main
Choose a base branch
from
bb/title-summarizer
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,213 @@ | ||
| import { describe, it, expect, vi, beforeEach, afterEach } from "vitest" | ||
| import { TitleSummarizer } from "../titleSummarizer" | ||
| import type { ProviderSettings, ProviderSettingsEntry } from "@roo-code/types" | ||
| import { TelemetryService } from "@roo-code/telemetry" | ||
| import { singleCompletionHandler } from "../../../utils/single-completion-handler" | ||
|
|
||
| // Mock TelemetryService | ||
| vi.mock("@roo-code/telemetry", () => ({ | ||
| TelemetryService: { | ||
| instance: { | ||
| captureEvent: vi.fn(), | ||
| }, | ||
| }, | ||
| })) | ||
|
|
||
| // Mock singleCompletionHandler | ||
| vi.mock("../../../utils/single-completion-handler", () => ({ | ||
| singleCompletionHandler: vi.fn(), | ||
| })) | ||
|
|
||
| describe("TitleSummarizer", () => { | ||
| const mockApiConfiguration: ProviderSettings = { | ||
| apiProvider: "anthropic", | ||
| apiKey: "test-key", | ||
| apiModelId: "claude-3-opus-20240229", | ||
| } | ||
|
|
||
| const mockListApiConfigMeta: ProviderSettingsEntry[] = [ | ||
| { | ||
| id: "default", | ||
| name: "Default", | ||
| apiProvider: "anthropic", | ||
| }, | ||
| { | ||
| id: "enhancement", | ||
| name: "Enhancement", | ||
| apiProvider: "openai", | ||
| }, | ||
| ] | ||
|
|
||
| const mockProviderSettingsManager = { | ||
| getProfile: vi.fn().mockResolvedValue({ | ||
| id: "enhancement", | ||
| name: "Enhancement", | ||
| apiProvider: "openai", | ||
| openAiApiKey: "test-openai-key", | ||
| openAiModelId: "gpt-4", | ||
| }), | ||
| } as any // Mock the ProviderSettingsManager type | ||
|
|
||
| beforeEach(() => { | ||
| vi.clearAllMocks() | ||
| // Set default mock behavior | ||
| vi.mocked(singleCompletionHandler).mockResolvedValue("Short concise title") | ||
| }) | ||
|
|
||
| afterEach(() => { | ||
| vi.restoreAllMocks() | ||
| }) | ||
|
|
||
| describe("summarizeTitle", () => { | ||
| it("should successfully summarize a long title", async () => { | ||
| const longText = | ||
| "I need help implementing a comprehensive user authentication system with OAuth2 support for Google, Facebook, and GitHub providers, including secure session management, password reset functionality, email verification, two-factor authentication, and proper error handling with rate limiting to prevent brute force attacks" | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: longText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(true) | ||
| expect(result.summarizedTitle).toBe("Short concise title") | ||
| expect(result.summarizedTitle!.length).toBeLessThan(longText.length) | ||
| }) | ||
|
|
||
| it("should return original text if it's already short", async () => { | ||
| const shortText = "Fix bug in login" | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: shortText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(true) | ||
| // Text is already shorter than max length, so it returns as-is | ||
| expect(result.summarizedTitle).toBe(shortText) | ||
| }) | ||
|
|
||
| it("should use enhancement API configuration when provided", async () => { | ||
| const longText = | ||
| "This is a very long title that definitely needs summarization to be more concise and readable for users, making it easier to understand the main point of the task at hand" | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: longText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| enhancementApiConfigId: "enhancement", | ||
| listApiConfigMeta: mockListApiConfigMeta, | ||
| providerSettingsManager: mockProviderSettingsManager, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| // The function will call providerSettingsManager.getProfile if all conditions are met | ||
| expect(mockProviderSettingsManager.getProfile).toHaveBeenCalledWith({ id: "enhancement" }) | ||
| expect(result.success).toBe(true) | ||
| expect(result.summarizedTitle).toBe("Short concise title") | ||
| }) | ||
|
|
||
| it("should handle API errors gracefully", async () => { | ||
| const longText = | ||
| "This is a very long text that definitely needs summarization to be more concise and readable for users, making it easier to understand the main point of the task at hand" | ||
| vi.mocked(singleCompletionHandler).mockRejectedValueOnce(new Error("API Error")) | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: longText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(false) | ||
| expect(result.error).toBe("API Error") | ||
| expect(result.summarizedTitle).toBe(longText) | ||
| }) | ||
|
|
||
| it("should handle missing API configuration", async () => { | ||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: "Some text", | ||
| apiConfiguration: undefined as any, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(false) | ||
| expect(result.error).toBe("No API configuration available") | ||
| expect(result.summarizedTitle).toBe("Some text") | ||
| }) | ||
|
|
||
| it("should respect custom maxLength parameter", async () => { | ||
| const shortText = "Short text" | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: shortText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| maxLength: 100, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(true) | ||
| // Text is already shorter than maxLength, returns as-is | ||
| expect(result.summarizedTitle).toBe(shortText) | ||
| }) | ||
|
|
||
| it("should use custom support prompts when provided", async () => { | ||
| const customPrompts = { | ||
| SUMMARIZE_TITLE: "Custom summarization prompt: {{userInput}} (max 150 chars)", | ||
| } | ||
|
|
||
| // Short text doesn't need summarization | ||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: "Text to summarize", | ||
| apiConfiguration: mockApiConfiguration, | ||
| customSupportPrompts: customPrompts, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(true) | ||
| expect(result.summarizedTitle).toBe("Text to summarize") | ||
|
Comment on lines
+152
to
+166
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This test doesn't actually verify custom support prompts functionality. The text "Text to summarize" (18 characters) is shorter than To properly test custom support prompts: const customPrompts = {
SUMMARIZE_TITLE: "Custom summarization prompt: {{userInput}} (max 150 chars)",
}
const longText = "a".repeat(151)
const result = await TitleSummarizer.summarizeTitle({
text: longText,
apiConfiguration: mockApiConfiguration,
customSupportPrompts: customPrompts,
maxLength: 150,
})
// Verify the custom prompt was actually used
expect(singleCompletionHandler).toHaveBeenCalledWith(
mockApiConfiguration,
expect.stringContaining("Custom summarization prompt")
) |
||
| }) | ||
|
|
||
| it("should handle empty response from API", async () => { | ||
| const longText = | ||
| "This is a very long text that definitely needs summarization to be more concise and readable for users, making it easier to understand the main point of the task at hand" | ||
| vi.mocked(singleCompletionHandler).mockResolvedValueOnce("") | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: longText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(false) | ||
| expect(result.error).toBe("Received empty summarized title") | ||
| expect(result.summarizedTitle).toBe(longText) | ||
| }) | ||
|
|
||
| it("should trim whitespace from summarized title", async () => { | ||
| const longText = | ||
| "This is a very long text that definitely needs summarization to be more concise and readable for users, making it easier to understand the main point of the task at hand" | ||
| vi.mocked(singleCompletionHandler).mockResolvedValueOnce(" Trimmed title \n") | ||
|
|
||
| const result = await TitleSummarizer.summarizeTitle({ | ||
| text: longText, | ||
| apiConfiguration: mockApiConfiguration, | ||
| maxLength: 150, | ||
| }) | ||
|
|
||
| expect(result.success).toBe(true) | ||
| expect(result.summarizedTitle).toBe("Trimmed title") | ||
| }) | ||
| }) | ||
|
|
||
| describe("captureTelemetry", () => { | ||
| it("should not capture telemetry events (currently disabled)", () => { | ||
| const taskId = "test-task-123" | ||
| const originalLength = 250 | ||
| const summarizedLength = 100 | ||
|
|
||
| TitleSummarizer.captureTelemetry(taskId, originalLength, summarizedLength) | ||
|
|
||
| // Since telemetry is commented out in the implementation, it should not be called | ||
| expect(TelemetryService.instance.captureEvent).not.toHaveBeenCalled() | ||
| }) | ||
| }) | ||
| }) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test has incorrect expectations. When
textis "Some text" (9 characters) andmaxLengthis 150, the early return on line 46 oftitleSummarizer.tswill execute, returning{ success: true, summarizedTitle: text }without ever validating the API configuration.The test expects
success: falseand an error about missing API configuration, but this validation never occurs due to the short text length.To properly test missing API configuration handling, use text longer than
maxLengthto bypass the early return:Note: The error message should also be "No valid API configuration provided" (from
singleCompletionHandler.ts:14) not "No API configuration available".