feat: Improve logging for debugging and monitoring by thisisharsh7 · Pull Request #118 · jentic/standard-agent

thisisharsh7 · 2025-10-09T19:10:37Z

Summary

Adds structured logging to key decision points throughout the agent execution flow.

Fixes #84

Summary by CodeRabbit

Release Notes

Chores
- Enhanced internal logging and instrumentation across agent components to improve system observability and facilitate debugging.
- Strengthened input validation in agent tools for better error handling and robustness.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

thisisharsh7 · 2025-10-09T19:10:58Z

kindly review it and let me know for the changes. thanks!

thisisharsh7 · 2025-10-18T17:00:42Z

hey @rishikesh-jentic can you please let me know if there is any change needed?

rishikesh-jentic · 2026-01-14T17:45:16Z

@CodeRabbit review this PR

coderabbitai · 2026-01-14T17:45:31Z

@rishikesh-jentic: I'll review the changes in this PR that improve logging for debugging and monitoring.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai · 2026-01-14T17:45:36Z

📝 Walkthrough

Walkthrough

This pull request adds structured logging instrumentation across the agent framework to enhance observability. Logs capture initialization events, LLM interactions with token usage metrics, goal preprocessing activities, and tool execution details without modifying control flow or function signatures.

Changes

Cohort / File(s)	Summary
Reasoner initialization logging `agents/reasoner/react.py`, `agents/reasoner/rewoo.py`	Added debug logs in constructors to emit `react_reasoner_initialized` and `rewoo_reasoner_initialized` events with configuration parameters (max_turns/iterations, retries, top_k).
LLM interaction logging `agents/llm/base_llm.py`, `agents/llm/litellm.py`	Added instrumentation around prompt/response cycles: pre-call logs capture model and content preview; post-response logs record token usage and response metrics. Enhanced exception handling in token extraction.
Agent and goal preprocessing logging `agents/standard_agent.py`, `agents/goal_preprocessor/conversational.py`	Added logs for agent lifecycle (initialization, solve start, goal preview) and goal preprocessing steps. Logs intervention events from goal preprocessor with preserved context.
Tool execution and validation `agents/tools/jentic.py`	Added type guards to enforce JenticTool types in load/execute methods. Introduced logging for search completion (result count) and successful tool execution with output length.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🐰 Wiggles nose with delight

Now logs illuminate each step with care,
From init to reasoning, information everywhere,
The agents whisper their secrets so bright,
Each token, each turn, now visible in sight! 🥕✨
Better debugging hops into the night!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 53.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the primary objective of the PR: adding logging instrumentation throughout the codebase for debugging and monitoring purposes.
Linked Issues check	✅ Passed	The PR successfully addresses all coding-related acceptance criteria from issue `#84`: adds logging to key decision points, follows structured logging patterns, maintains reasonable verbosity, and introduces no performance impact.
Out of Scope Changes check	✅ Passed	All changes are within scope: logging enhancements in goal_preprocessor, llm base and implementation, reasoners, standard_agent, and jentic tools align with issue `#84` objectives.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

🧹 Recent nitpick comments

agents/tools/jentic.py (2)
126-132: Consider removing redundant found_tools field.

The found_tools boolean is redundant since it can be trivially inferred from result_count > 0. This adds minor noise to the logs.
♻️ Suggested simplification
         logger.info("tool_search_complete",
                     query=query,
-                    result_count=len(results),
-                    found_tools=len(results) > 0)
+                    result_count=len(results))
175-178: Potential performance concern with str() conversion on large outputs.

Converting result.output to string just to measure length could be expensive for large or complex outputs (e.g., deeply nested dicts). Consider a safer approach.
♻️ Safer length calculation
             logger.info("tool_execution_success",
                         tool_id=tool.id,
-                        output_length=len(str(result.output)) if result.output else 0)
+                        has_output=result.output is not None)
Alternatively, if output size is valuable for monitoring, consider using sys.getsizeof() or checking type first:
import sys
# ...
output_size = sys.getsizeof(result.output) if result.output else 0

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f3b76b3 and 891c2cb.

📒 Files selected for processing (7)

agents/goal_preprocessor/conversational.py
agents/llm/base_llm.py
agents/llm/litellm.py
agents/reasoner/react.py
agents/reasoner/rewoo.py
agents/standard_agent.py
agents/tools/jentic.py

🧰 Additional context used

🧬 Code graph analysis (3)

agents/goal_preprocessor/conversational.py (1)

tests/utils/test_logger.py (1)

debug (220-221)

agents/llm/litellm.py (1)

tests/utils/test_logger.py (1)

debug (220-221)

agents/llm/base_llm.py (2)

tests/utils/test_logger.py (1)

debug (220-221)

agents/llm/litellm.py (1)

completion (23-63)

🪛 Ruff (0.14.11)

agents/llm/litellm.py

147-147: Do not catch blind exception: Exception

(BLE001)

🔇 Additional comments (11)

agents/reasoner/react.py (1)

35-38: LGTM!

The initialization log follows the existing structured logging pattern and captures useful configuration parameters for debugging. The snake_case event name is consistent with other events in the codebase.

agents/reasoner/rewoo.py (1)

78-82: LGTM!

The initialization log is consistent with the pattern in react.py and captures relevant configuration parameters for debugging the ReWOO reasoner.

agents/goal_preprocessor/conversational.py (1)

37-39: LGTM!

Using debug level for entry-point tracing is appropriate. Since user goals could potentially contain sensitive information, the debug level ensures this detailed logging is typically disabled in production while remaining available for local debugging.

agents/llm/litellm.py (2)

52-56: LGTM!

Token usage metrics are valuable for monitoring and cost tracking. Debug level is appropriate for this high-frequency operational data.

147-149: LGTM - Broad exception catch is acceptable here.

The static analysis warning (BLE001) about catching Exception is a valid general concern, but in this context it's appropriate. This is a best-effort token extraction utility that handles varied provider response formats. Graceful degradation to (None, None, None) is the correct behavior, and logging the error at debug level aids troubleshooting without disrupting the main flow.

agents/llm/base_llm.py (2)

130-139: LGTM!

Good security-conscious logging design:

Prompt preview is truncated to 100 characters, avoiding full prompt logging

Response logging captures only metadata (length, token counts), not the actual content

160-168: LGTM!

Logging only the response keys (not values) is a sensible approach for JSON responses, providing useful debugging information without risking exposure of potentially sensitive response data.

agents/standard_agent.py (3)

83-88: LGTM! Well-structured initialization logging.

The structured logging captures key configuration details that are useful for debugging without being verbose. Good use of getattr to safely access model attribute.

100-102: LGTM! Appropriate solve-start logging with truncation.

Good practice to truncate goal_preview to 100 characters to avoid log bloat while still providing useful context. The run_id enables correlation with the final_result log.

104-111: LGTM! Good use of DEBUG vs INFO for preprocessing logs.

Appropriate log levels: DEBUG for routine preprocessing start (line 105), INFO for intervention events that change control flow (line 108). This keeps logs clean while surfacing important events.

agents/tools/jentic.py (1)

139-140: LGTM! Type guards are good defensive programming.

The type guards in load() and execute() provide clear error messages when called with incorrect tool types. This is helpful for debugging integration issues.

Also applies to: 158-159

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

rishikesh-jentic · 2026-01-16T18:25:34Z

agents/goal_preprocessor/conversational.py


    @observe()
    def process(self, goal: str, history: Sequence[Dict[str, Any]]) -> Tuple[str, str | None]:
+        logger.debug("goal_preprocessor_entry",


Suggested change

logger.debug("goal_preprocessor_entry",

logger.debug("goal_preprocessor_entry", goal=goal, history_length=len(history))

rishikesh-jentic · 2026-01-16T18:26:51Z

agents/llm/base_llm.py

        Returns:
            The assistant's response text.
        """
+        logger.debug("llm_prompt",


Suggested change

logger.debug("llm_prompt",

logger.debug("llm_prompt", model=self.model, prompt_preview=content[:100] + "..." if len(content) > 100 else content, kwargs=kwargs)

rishikesh-jentic · 2026-01-16T18:27:46Z

agents/llm/base_llm.py

+                     prompt_preview=content[:100] + "..." if len(content) > 100 else content,
+                     kwargs=kwargs)
        resp = self.completion([{"role": "user", "content": content}], **kwargs)
+        logger.debug("llm_response",


Suggested change

logger.debug("llm_response",

logger.debug("llm_response", model=self.model, response_length=len(resp.text), prompt_tokens=resp.prompt_tokens, completion_tokens=resp.completion_tokens)

rishikesh-jentic · 2026-01-16T18:29:02Z

agents/llm/litellm.py


        prompt_tokens, completion_tokens, total_tokens = self._extract_token_usage(resp)

+        logger.debug("llm_completion_success",


Suggested change

logger.debug("llm_completion_success",

logger.debug("llm_completion_success", model=self.model, prompt_tokens=prompt_tokens, completion_tokens=completion_tokens, total_tokens=total_tokens)

rishikesh-jentic · 2026-01-16T18:29:17Z

agents/reasoner/react.py

        self.max_turns = max_turns
        self.top_k = top_k

+        logger.info("react_reasoner_initialized",


Suggested change

logger.info("react_reasoner_initialized",

logger.info("react_reasoner_initialized", max_turns=self.max_turns, top_k=self.top_k)

rishikesh-jentic · 2026-01-16T18:29:35Z

agents/reasoner/rewoo.py

        self.max_retries = max_retries
        self.top_k = top_k

+        logger.info("rewoo_reasoner_initialized",


Suggested change

logger.info("rewoo_reasoner_initialized",

logger.info("rewoo_reasoner_initialized", max_iterations=self.max_iterations, max_retries=self.max_retries, top_k=self.top_k)

rishikesh-jentic · 2026-01-16T18:30:16Z

agents/standard_agent.py

        run_id = uuid4().hex
        start_time = time.perf_counter()

+        logger.info("agent_solve_start",


Suggested change

logger.info("agent_solve_start",

logger.info("agent_solve_start", run_id=run_id, goal_preview=goal[:100] + "..." if len(goal) > 100 else goal)

rishikesh-jentic · 2026-01-16T18:30:35Z

agents/tools/jentic.py

-        return [JenticTool(result.model_dump(exclude_none=False)) for result in response.results] if response.results else []
+        results = [JenticTool(result.model_dump(exclude_none=False)) for result in response.results] if response.results else []
+
+        logger.info("tool_search_complete",


Suggested change

logger.info("tool_search_complete",

logger.info("tool_search_complete", query=query, result_count=len(results), found_tools=len(results) > 0)

rishikesh-jentic · 2026-01-16T18:30:49Z

agents/tools/jentic.py


                raise ToolExecutionError(result.error, tool)
+
+            logger.info("tool_execution_success",


Suggested change

logger.info("tool_execution_success",

logger.info("tool_execution_success", tool_id=tool.id, output_length=len(str(result.output)) if result.output else 0)

feat: add logging to key descision points

891c2cb

thisisharsh7 requested a review from a team as a code owner October 9, 2025 19:10

rishikesh-jentic reviewed Jan 16, 2026

View reviewed changes

	logger.debug("goal_preprocessor_entry",
	logger.debug("goal_preprocessor_entry", goal=goal, history_length=len(history))

	logger.debug("llm_prompt",
	logger.debug("llm_prompt", model=self.model, prompt_preview=content[:100] + "..." if len(content) > 100 else content, kwargs=kwargs)

	logger.debug("llm_response",
	logger.debug("llm_response", model=self.model, response_length=len(resp.text), prompt_tokens=resp.prompt_tokens, completion_tokens=resp.completion_tokens)


		prompt_tokens, completion_tokens, total_tokens = self._extract_token_usage(resp)

		logger.debug("llm_completion_success",

	logger.debug("llm_completion_success",
	logger.debug("llm_completion_success", model=self.model, prompt_tokens=prompt_tokens, completion_tokens=completion_tokens, total_tokens=total_tokens)

	logger.info("react_reasoner_initialized",
	logger.info("react_reasoner_initialized", max_turns=self.max_turns, top_k=self.top_k)

	logger.info("rewoo_reasoner_initialized",
	logger.info("rewoo_reasoner_initialized", max_iterations=self.max_iterations, max_retries=self.max_retries, top_k=self.top_k)

	logger.info("agent_solve_start",
	logger.info("agent_solve_start", run_id=run_id, goal_preview=goal[:100] + "..." if len(goal) > 100 else goal)

	logger.info("tool_search_complete",
	logger.info("tool_search_complete", query=query, result_count=len(results), found_tools=len(results) > 0)


		raise ToolExecutionError(result.error, tool)

		logger.info("tool_execution_success",

	logger.info("tool_execution_success",
	logger.info("tool_execution_success", tool_id=tool.id, output_length=len(str(result.output)) if result.output else 0)

Conversation

thisisharsh7 commented Oct 9, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Summary by CodeRabbit

Release Notes

Uh oh!

thisisharsh7 commented Oct 9, 2025

Uh oh!

thisisharsh7 commented Oct 18, 2025

Uh oh!

rishikesh-jentic commented Jan 14, 2026

Uh oh!

coderabbitai bot commented Jan 14, 2026

Uh oh!

coderabbitai bot commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

thisisharsh7 commented Oct 9, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Jan 14, 2026 •

edited

Loading