feat(prompts): enforce codebase_search as primary code understanding tool

- Add conditional codebase_search enforcement in tool use guidelines
- Modify objective section to prioritize codebase_search when available
- Update rules section with critical codebase_search-first rule
- Pass CodeIndexManager to prompt sections for availability checks
- Ensure graceful degradation when codebase_search is unavailable

Author: hannesrudolph

docs/codebase-search-enforcement.md

@@ -0,0 +1,91 @@
+# Codebase Search Enforcement Documentation
+
+## Overview
+
+This document describes the mechanisms implemented to ensure that the `codebase_search` tool is properly utilized as the first line of understanding when working with codebase context in Roo Code.
+
+## Implementation Details
+
+### 1. Tool Use Guidelines Section
+
+**File**: `src/core/prompts/sections/tool-use-guidelines.ts`
+
+The function now accepts a `CodeIndexManager` parameter to conditionally include codebase_search enforcement:
+- When codebase_search is available: **IMPORTANT: When starting a new task or when you need to understand existing code/functionality, you MUST use the `codebase_search` tool FIRST before any other search tools.**
+- When unavailable: Standard tool selection guidance without codebase_search enforcement
+- Maintains proper numbering regardless of availability
+
+### 2. Objective Section
+
+**File**: `src/core/prompts/sections/objective.ts`
+
+Enhanced the thinking process in step 3 with conditional enforcement:
+- When codebase_search is available: Requires using `codebase_search` tool first if the task involves understanding existing code or functionality
+- When unavailable: Proceeds directly to analyzing file structure
+- Integrated into the <thinking> tags analysis workflow
+
+### 3. Rules Section
+
+**File**: `src/core/prompts/sections/rules.ts`
+
+Added a conditional critical rule:
+- When codebase_search is available: **CRITICAL: When you need to understand existing code or functionality, ALWAYS use the `codebase_search` tool FIRST before using search_files or other file exploration tools.**
+- When unavailable: No codebase_search rule is included
+- search_files guidance adjusts accordingly (mentions "after codebase_search" only when available)
+
+### 4. Mode-Specific Instructions
+
+**File**: `src/shared/modes.ts`
+
+Updated the architect mode's custom instructions:
+- Modified step 1 to explicitly state: **ALWAYS start with the `codebase_search` tool (if available) to understand existing functionality and code structure before using other tools like read_file or search_files.**
+- This ensures that even in architect mode, which is focused on planning, the codebase_search tool is prioritized when available
+
+### 5. System Prompt Integration
+
+**File**: `src/core/prompts/system.ts`
+
+Updated to pass `CodeIndexManager` to the relevant sections:
+- `getToolUseGuidelinesSection(codeIndexManager)`
+- `getObjectiveSection(codeIndexManager)`
+- `getRulesSection(cwd, supportsComputerUse, effectiveDiffStrategy, codeIndexManager)`
+
+## How It Works
+
+The enforcement works through multiple layers:
+
+1. **Conditional Availability Check**: The system checks if `CodeIndexManager` is enabled, configured, and initialized before including codebase_search guidance.
+
+2. **System Prompt Level**: The tool use guidelines and objective sections conditionally include codebase_search enforcement based on availability.
+
+3. **Mode-Specific Level**: Individual modes (like architect) have their custom instructions updated to reinforce the codebase_search-first approach when available.
+
+4. **User Custom Instructions**: The user's global instruction "Always use codebase_search before any other search first" provides an additional layer of enforcement.
+
+## Benefits
+
+1. **Better Context Understanding**: By using semantic search first, the AI can find functionally relevant code even without knowing exact keywords or file names.
+
+2. **Efficiency**: Reduces the need for multiple regex searches or file explorations by finding the most relevant code upfront.
+
+3. **Consistency**: Ensures a standardized approach to understanding codebase context across all modes and tasks.
+
+4. **Graceful Degradation**: The system only mentions codebase_search when it's actually available, preventing confusion when the feature is disabled.
+
+## Testing
+
+Test files have been created to verify the implementation:
+- `src/core/prompts/sections/__tests__/tool-use-guidelines.test.ts`
+- `src/core/prompts/sections/__tests__/objective.test.ts`
+
+These tests verify both scenarios:
+- When codebase_search is available (enforcement is included)
+- When codebase_search is not available (enforcement is excluded)
+
+## Future Considerations
+
+1. The enforcement only applies when the `codebase_search` tool is available (i.e., when the code index feature is enabled, configured, and initialized).
+
+2. The system gracefully falls back to other search methods if `codebase_search` is not available, without mentioning it in the prompts.
+
+3. The enforcement is designed to guide behavior without being overly restrictive - it emphasizes "MUST" for new tasks and understanding existing code, but allows flexibility for other scenarios.

src/core/prompts/__tests__/__snapshots__/system.test.ts.snap

@@ -0,0 +1,77 @@
+import { getObjectiveSection } from "../objective"
+import { CodeIndexManager } from "../../../../services/code-index/manager"
+
+describe("getObjectiveSection", () => {
+	// Mock CodeIndexManager with codebase search available
+	const mockCodeIndexManagerEnabled = {
+		isFeatureEnabled: true,
+		isFeatureConfigured: true,
+		isInitialized: true,
+	} as CodeIndexManager
+
+	// Mock CodeIndexManager with codebase search unavailable
+	const mockCodeIndexManagerDisabled = {
+		isFeatureEnabled: false,
+		isFeatureConfigured: false,
+		isInitialized: false,
+	} as CodeIndexManager
+
+	describe("when codebase_search is available", () => {
+		it("should include codebase_search first enforcement in thinking process", () => {
+			const objective = getObjectiveSection(mockCodeIndexManagerEnabled)
+			
+			// Check that the objective includes the codebase_search enforcement
+			expect(objective).toContain("if the task involves understanding existing code or functionality, you MUST use the `codebase_search` tool")
+			expect(objective).toContain("BEFORE using any other search or file exploration tools")
+		})
+	})
+
+	describe("when codebase_search is not available", () => {
+		it("should not include codebase_search enforcement", () => {
+			const objective = getObjectiveSection(mockCodeIndexManagerDisabled)
+			
+			// Check that the objective does not include the codebase_search enforcement
+			expect(objective).not.toContain("you MUST use the `codebase_search` tool")
+			expect(objective).not.toContain("BEFORE using any other search or file exploration tools")
+		})
+	})
+
+	it("should maintain proper structure regardless of codebase_search availability", () => {
+		const objectiveEnabled = getObjectiveSection(mockCodeIndexManagerEnabled)
+		const objectiveDisabled = getObjectiveSection(mockCodeIndexManagerDisabled)
+		
+		// Check that all numbered items are present in both cases
+		for (const objective of [objectiveEnabled, objectiveDisabled]) {
+			expect(objective).toContain("1. Analyze the user's task")
+			expect(objective).toContain("2. Work through these goals sequentially")
+			expect(objective).toContain("3. Remember, you have extensive capabilities")
+			expect(objective).toContain("4. Once you've completed the user's task")
+			expect(objective).toContain("5. The user may provide feedback")
+		}
+	})
+
+	it("should include thinking tags guidance regardless of codebase_search availability", () => {
+		const objectiveEnabled = getObjectiveSection(mockCodeIndexManagerEnabled)
+		const objectiveDisabled = getObjectiveSection(mockCodeIndexManagerDisabled)
+		
+		// Check that thinking tags guidance is included in both cases
+		for (const objective of [objectiveEnabled, objectiveDisabled]) {
+			expect(objective).toContain("<thinking></thinking> tags")
+			expect(objective).toContain("analyze the file structure provided in environment_details")
+			expect(objective).toContain("think about which of the provided tools is the most relevant")
+		}
+	})
+
+	it("should include parameter inference guidance regardless of codebase_search availability", () => {
+		const objectiveEnabled = getObjectiveSection(mockCodeIndexManagerEnabled)
+		const objectiveDisabled = getObjectiveSection(mockCodeIndexManagerDisabled)
+		
+		// Check parameter inference guidance in both cases
+		for (const objective of [objectiveEnabled, objectiveDisabled]) {
+			expect(objective).toContain("Go through each of the required parameters")
+			expect(objective).toContain("determine if the user has directly provided or given enough information to infer a value")
+			expect(objective).toContain("DO NOT invoke the tool (not even with fillers for the missing params)")
+			expect(objective).toContain("ask_followup_question tool")
+		}
+	})
+})

src/core/prompts/sections/__tests__/objective.test.ts

@@ -0,0 +1,78 @@
+import { getToolUseGuidelinesSection } from "../tool-use-guidelines"
+import { CodeIndexManager } from "../../../../services/code-index/manager"
+
+describe("getToolUseGuidelinesSection", () => {
+	// Mock CodeIndexManager with codebase search available
+	const mockCodeIndexManagerEnabled = {
+		isFeatureEnabled: true,
+		isFeatureConfigured: true,
+		isInitialized: true,
+	} as CodeIndexManager
+
+	// Mock CodeIndexManager with codebase search unavailable
+	const mockCodeIndexManagerDisabled = {
+		isFeatureEnabled: false,
+		isFeatureConfigured: false,
+		isInitialized: false,
+	} as CodeIndexManager
+
+	describe("when codebase_search is available", () => {
+		it("should include codebase_search first enforcement", () => {
+			const guidelines = getToolUseGuidelinesSection(mockCodeIndexManagerEnabled)
+			
+			// Check that the guidelines include the codebase_search enforcement
+			expect(guidelines).toContain("IMPORTANT: When starting a new task or when you need to understand existing code/functionality, you MUST use the `codebase_search` tool FIRST")
+			expect(guidelines).toContain("before any other search tools")
+			expect(guidelines).toContain("semantic search tool helps you find relevant code based on meaning rather than just keywords")
+		})
+
+		it("should maintain proper numbering with codebase_search", () => {
+			const guidelines = getToolUseGuidelinesSection(mockCodeIndexManagerEnabled)
+			
+			// Check that all numbered items are present
+			expect(guidelines).toContain("1. In <thinking> tags")
+			expect(guidelines).toContain("2. **IMPORTANT:")
+			expect(guidelines).toContain("3. Choose the most appropriate tool")
+			expect(guidelines).toContain("3. If multiple actions are needed")
+			expect(guidelines).toContain("4. Formulate your tool use")
+			expect(guidelines).toContain("5. After each tool use")
+			expect(guidelines).toContain("6. ALWAYS wait for user confirmation")
+		})
+	})
+
+	describe("when codebase_search is not available", () => {
+		it("should not include codebase_search enforcement", () => {
+			const guidelines = getToolUseGuidelinesSection(mockCodeIndexManagerDisabled)
+			
+			// Check that the guidelines do not include the codebase_search enforcement
+			expect(guidelines).not.toContain("IMPORTANT: When starting a new task or when you need to understand existing code/functionality, you MUST use the `codebase_search` tool FIRST")
+			expect(guidelines).not.toContain("semantic search tool helps you find relevant code based on meaning")
+		})
+
+		it("should maintain proper numbering without codebase_search", () => {
+			const guidelines = getToolUseGuidelinesSection(mockCodeIndexManagerDisabled)
+			
+			// Check that all numbered items are present with correct numbering
+			expect(guidelines).toContain("1. In <thinking> tags")
+			expect(guidelines).toContain("2. Choose the most appropriate tool")
+			expect(guidelines).toContain("3. If multiple actions are needed")
+			expect(guidelines).toContain("4. Formulate your tool use")
+			expect(guidelines).toContain("5. After each tool use")
+			expect(guidelines).toContain("6. ALWAYS wait for user confirmation")
+		})
+	})
+
+	it("should include iterative process guidelines regardless of codebase_search availability", () => {
+		const guidelinesEnabled = getToolUseGuidelinesSection(mockCodeIndexManagerEnabled)
+		const guidelinesDisabled = getToolUseGuidelinesSection(mockCodeIndexManagerDisabled)
+		
+		// Check that the iterative process section is included in both cases
+		for (const guidelines of [guidelinesEnabled, guidelinesDisabled]) {
+			expect(guidelines).toContain("It is crucial to proceed step-by-step")
+			expect(guidelines).toContain("1. Confirm the success of each step before proceeding")
+			expect(guidelines).toContain("2. Address any issues or errors that arise immediately")
+			expect(guidelines).toContain("3. Adapt your approach based on new information")
+			expect(guidelines).toContain("4. Ensure that each action builds correctly")
+		}
+	})
+})

src/core/prompts/sections/__tests__/tool-use-guidelines.test.ts

@@ -1,4 +1,15 @@
-export function getObjectiveSection(): string {
+import { CodeIndexManager } from "../../../services/code-index/manager"
+
+export function getObjectiveSection(codeIndexManager?: CodeIndexManager): string {
+	const isCodebaseSearchAvailable = codeIndexManager &&
+		codeIndexManager.isFeatureEnabled &&
+		codeIndexManager.isFeatureConfigured &&
+		codeIndexManager.isInitialized
+
+	const codebaseSearchInstruction = isCodebaseSearchAvailable
+		? "First, if the task involves understanding existing code or functionality, you MUST use the `codebase_search` tool to search for relevant code based on the task's intent BEFORE using any other search or file exploration tools. Then, "
+		: "First, "
+
 	return `====
 
 OBJECTIVE
@@ -7,7 +18,7 @@ You accomplish a given task iteratively, breaking it down into clear steps and w
 
 1. Analyze the user's task and set clear, achievable goals to accomplish it. Prioritize these goals in a logical order.
 2. Work through these goals sequentially, utilizing available tools one at a time as necessary. Each goal should correspond to a distinct step in your problem-solving process. You will be informed on the work completed and what's remaining as you go.
-3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within <thinking></thinking> tags. First, analyze the file structure provided in environment_details to gain context and insights for proceeding effectively. Then, think about which of the provided tools is the most relevant tool to accomplish the user's task. Next, go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool use. BUT, if one of the values for a required parameter is missing, DO NOT invoke the tool (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters using the ask_followup_question tool. DO NOT ask for more information on optional parameters if it is not provided.
+3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within <thinking></thinking> tags. ${codebaseSearchInstruction}analyze the file structure provided in environment_details to gain context and insights for proceeding effectively. Next, think about which of the provided tools is the most relevant tool to accomplish the user's task. Go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool use. BUT, if one of the values for a required parameter is missing, DO NOT invoke the tool (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters using the ask_followup_question tool. DO NOT ask for more information on optional parameters if it is not provided.
 4. Once you've completed the user's task, you must use the attempt_completion tool to present the result of the task to the user. You may also provide a CLI command to showcase the result of your task; this can be particularly useful for web development tasks, where you can run e.g. \`open index.html\` to show the website you've built.
 5. The user may provide feedback, which you can use to make improvements and try again. But DO NOT continue in pointless back and forth conversations, i.e. don't end your responses with questions or offers for further assistance.`
 }