add thc

[dogancanbakir]

closes #1684

$ go run . -s thc -d hackerone.com       

               __    _____           __         
   _______  __/ /_  / __(_)___  ____/ /__  _____
  / ___/ / / / __ \/ /_/ / __ \/ __  / _ \/ ___/
 (__  ) /_/ / /_/ / __/ / / / / /_/ /  __/ /    
/____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/

                projectdiscovery.io

[INF] Current subfinder version v2.10.1 (latest)
[INF] Loading provider config from /Users/dogancanbakir/Library/Application Support/subfinder/provider-config.yaml
[INF] Enumerating subdomains for hackerone.com
api.hackerone.com
docs.hackerone.com
mta-sts.forwarding.hackerone.com
mta-sts.managed.hackerone.com
websockets.hackerone.com
www.hackerone.com
gslink.hackerone.com
mta-sts.hackerone.com
a.ns.hackerone.com
b.ns.hackerone.com
support.hackerone.com
[INF] Found 11 subdomains for hackerone.com in 461 milliseconds 956 microseconds

Summary by CodeRabbit

• New Features
  • Integrated THC as a new passive data source for subdomain reconnaissance
  • Provides automated API-based subdomain discovery and enumeration
  • Implements intelligent pagination to ensure complete result collection across large datasets
  • Enhances passive intelligence gathering capabilities without requiring API credentials
  • Seamlessly extends existing passive reconnaissance workflow with additional data source coverage

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

A new passive subdomain discovery source for ip.thc.org is introduced. The THC source type is implemented with standard interface methods, registered in the passive sources list, and corresponding test fixtures are updated to reflect the additional source.

Changes

Cohort / File(s)	Summary
THC Source Implementation pkg/subscraping/sources/thc/thc.go	New source implementation with Run method for iterative API-based subdomain discovery via https://ip.thc.org/api/v1/lookup/subdomains, including pagination support, result aggregation, and statistics tracking. Implements standard Source interface methods (Name, IsDefault, HasRecursiveSupport, NeedsKey, AddApiKeys, Statistics).
Source Registration & Tests pkg/passive/sources.go, pkg/passive/sources_test.go	Registers THC source in AllSources slice and updates test fixtures to include the new source in default and all-sources-derived groups.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• API integration verification: Validate the JSON request/response handling and pagination logic against the actual ip.thc.org API behavior
• Error handling: Ensure proper handling of API errors and edge cases in the Run loop
• Statistics tracking: Verify that timing and result counts are correctly recorded

Poem

🐰 A THC source hops in so bright,
Subdomain discovery through the night,
With pagination dancing free,
More data sources, one, two, three!
Integration smooth—the sources agree! 🌟

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'add thc' is vague and overly concise; it lacks specificity about what 'thc' refers to or why it matters, making it unclear without additional context.	Use a more descriptive title like 'Add THC passive subdomain source' to clearly indicate this integrates a new passive data source for subdomain enumeration.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	The PR successfully implements integration of the THC passive subdomain source by adding a new Source type with the required interface methods, registering it in AllSources, and updating tests accordingly.
Out of Scope Changes check	✅ Passed	All changes are directly related to adding the THC source: the new implementation file, source registration, and test updates; no unrelated modifications detected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch add_thc

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7842ebf and 692e30a.

📒 Files selected for processing (3)

• pkg/passive/sources.go (2 hunks)
• pkg/passive/sources_test.go (2 hunks)
• pkg/subscraping/sources/thc/thc.go (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (2)

pkg/passive/sources.go (1)

pkg/subscraping/sources/thc/thc.go (1)

• Source (23-28)

pkg/subscraping/sources/thc/thc.go (1)

pkg/subscraping/types.go (2)

• Session (71-78)
• Statistics (29-34)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)

• GitHub Check: Test Builds (windows-latest)
• GitHub Check: Test Builds (ubuntu-latest)
• GitHub Check: Test Builds (macOS-latest)
• GitHub Check: Analyze (go)
• GitHub Check: release-test

🔇 Additional comments (9)

pkg/subscraping/sources/thc/thc.go (6)

1-34: LGTM! Clean type definitions.

The package structure, imports, and type definitions are well-organized and appropriate for the THC API integration.

---

37-51: LGTM! Proper initialization and timing setup.

The channel creation, counter initialization, and deferred timing logic follow the expected pattern for this codebase.

---

83-86: LGTM! Result processing is correct.

The loop correctly extracts domains from the response and yields them through the results channel with proper source attribution.

---

88-92: LGTM! Pagination logic is sound.

The pagination control correctly updates the page state and terminates when no more pages are available.

---

99-127: LGTM! Interface implementation is complete and correct.

All required Source interface methods are properly implemented with appropriate return values for the THC source characteristics (default source, no API key required, no recursive support).

---

66-72: The concern about nil pointer dereference is unfounded. The DiscardHTTPResponse method in agent.go already includes a nil check: if response != nil at line 116. The response is safely handled even when session.Post() returns an error and a nil response.

Likely an incorrect or invalid review comment.

pkg/passive/sources.go (2)

51-51: LGTM! Import added correctly.

The THC package import is properly placed in alphabetical order with other source imports.

---

112-112: LGTM! Source registration is correct.

The THC source is properly instantiated and added to the AllSources array, enabling it for passive enumeration.

pkg/passive/sources_test.go (1)

63-63: LGTM! Test expectations updated correctly.

The test fixtures properly include "thc" in both expectedAllSources and expectedDefaultSources lists, which aligns with the source's IsDefault() returning true.

Also applies to: 104-104

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Mzack9999]

lgtm!