fix body read err

[dogancanbakir]

closes #708

Summary by CodeRabbit

Bug Fixes

• Improved HTTP response handling in agent operations to enhance reliability and system stability.
• Optimized error handling and resource management across agent modules for more robust query execution.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Fixed HTTP response body handling in FOFA and Hunter query agents by adding deferred body closure and simplifying error handling to remove redundant raw payload emission on JSON decode failures.

Changes

Cohort / File(s)	Summary
HTTP Response Management sources/agent/fofa/fofa.go, sources/agent/hunter/hunter.go	Added deferred close of HTTP response body in query functions to ensure proper resource cleanup. Removed previous behavior of reading entire response body into RespBodyByBodyBytes and emitting Raw payload on decode errors. On JSON decode failure, now directly return Result with error instead of constructing raw payload.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

• Verify deferred resp.Body.Close() is correctly positioned and error logging works as intended
• Confirm that removal of raw payload emission on decode errors doesn't inadvertently discard needed diagnostic data
• Check that error result handling maintains proper error propagation to callers

Poem

🐰 With defer we close, no leaks shall pass,
HTTP bodies rest, clean and fast.
Raw payloads fade, errors shine clear,
FOFA queries return, results appear! ✨

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'fix body read err' is vague and abbreviates key terms, making it unclear what specific issue is being addressed without additional context.	Use a clearer, more descriptive title such as 'Fix HTTP response body handling in FOFA and Hunter agents' to better convey the actual changes.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	The PR directly addresses issue #708 by fixing HTTP response body handling in FOFA and Hunter agents, restoring query results that were broken after a regression.
Out of Scope Changes check	✅ Passed	All changes are focused on fixing response body handling in FOFA and Hunter agents; no unrelated modifications detected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch 708_fix_read_err

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (2)

sources/agent/hunter/hunter.go (1)

82-86: Consider using a more appropriate log level for body close errors.

Using Info() for an error condition is unconventional. Body close errors are typically low-priority but still indicate something went wrong. Consider using Warning() or Debug() depending on how visible you want this to be.

 	defer func(Body io.ReadCloser) {
 		if bodyCloseErr := Body.Close(); bodyCloseErr != nil {
-			gologger.Info().Msgf("response body close error : %v", bodyCloseErr)
+			gologger.Warning().Msgf("response body close error: %v", bodyCloseErr)
 		}
 	}(resp.Body)

sources/agent/fofa/fofa.go (1)

96-98: Simplified error handling on decode failures.

The removal of raw response body attachment on JSON decode errors simplifies the code but reduces debugging information for malformed API responses. This is acceptable for the current fix, though capturing a preview of the response body in the error message could help future troubleshooting.

If you want to enhance debugging without complexity, consider:

 if err := json.NewDecoder(resp.Body).Decode(fofaResponse); err != nil {
-	results <- sources.Result{Source: agent.Name(), Error: err}
+	results <- sources.Result{Source: agent.Name(), Error: fmt.Errorf("decode error: %w", err)}
 	return nil
 }

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b01669e and 5e53315.

📒 Files selected for processing (2)

• sources/agent/fofa/fofa.go (1 hunks)
• sources/agent/hunter/hunter.go (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (2)

sources/agent/fofa/fofa.go (2)

sources/agent/fofa/response.go (1)

• FofaResponse (4-12)

sources/result.go (1)

• Result (8-17)

sources/agent/hunter/hunter.go (2)

sources/agent/hunter/response.go (1)

• Response (17-21)

sources/result.go (1)

• Result (8-17)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)

• GitHub Check: Test Builds (macOS-latest, 1.24.x)
• GitHub Check: Test Builds (ubuntu-latest, 1.24.x)
• GitHub Check: Test Builds (windows-latest, 1.24.x)
• GitHub Check: release-test
• GitHub Check: Analyze (go)

🔇 Additional comments (2)

sources/agent/hunter/hunter.go (1)

88-92: LGTM!

Using json.NewDecoder(resp.Body).Decode() directly is the idiomatic approach for decoding JSON from HTTP responses. This avoids the intermediate buffer allocation and simplifies error handling. The removal of raw payload emission on decode errors is appropriate since malformed JSON data isn't useful to downstream consumers.

sources/agent/fofa/fofa.go (1)

89-93: Critical fix: Ensures response body stays open during decode.

This defer pattern correctly addresses the regression in issue #708. By deferring the body close until after the JSON decode completes (line 96), the decoder can now fully read the response body, fixing the "empty results" bug in v1.2.1.

The error logging on close failure is also good defensive practice.