Trawlingweb - new component

[michelle0927]

Resolves #10972

Summary by CodeRabbit

• New Features

  • Added search news functionality to the Trawlingweb integration, allowing users to search news with customizable parameters including query, date range, sorting options, and result limits.

• Chores

  • Updated component version and dependencies.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Preview	Comments	Updated (UTC)
pipedream-docs	Ignored			Oct 28, 2025 7:26pm
pipedream-docs-redirect-do-not-edit	Ignored			Oct 28, 2025 7:26pm

[coderabbitai]

Walkthrough

This PR adds a new Search News action to the Trawlingweb integration, refactors the API client to centralize HTTP request handling with automatic token injection, updates the package version and dependencies, effectively modularizing the integration layer.

Changes

Cohort / File(s)	Summary
New Search News Action components/trawlingweb/actions/search-news/search-news.mjs	Exports a new action for searching news via Trawlingweb with configurable query parameters (dateStart, dateEnd, sort, order, maxResults), runs the search via the client, and returns results with a summary.
Refactored API Client components/trawlingweb/trawlingweb.app.mjs	Introduces _baseUrl() and _makeRequest() helper methods for centralized request handling with automatic token injection; delegates searchNews() to _makeRequest(); removes authKeys() method.
Package Configuration components/trawlingweb/package.json	Version bumped from 0.0.1 to 0.1.0; adds dependency on @pipedream/platform@^3.1.0.

Sequence Diagram

sequenceDiagram
    participant Action as Search News Action
    participant App as Trawlingweb App
    participant HTTP as HTTP Client (axios)
    participant API as Trawlingweb API

    Action->>Action: Assemble parameters<br/>(query, dateStart, dateEnd, etc.)
    Action->>App: searchNews(opts)
    activate App
    App->>App: _makeRequest(opts)
    activate App
    App->>App: Inject _baseUrl() & token
    App->>HTTP: axios.request({<br/>baseURL, params, token...})
    activate HTTP
    HTTP->>API: POST /search/news
    activate API
    API-->>HTTP: results[]
    deactivate API
    HTTP-->>App: response data
    deactivate HTTP
    deactivate App
    App-->>Action: data array
    deactivate App
    Action->>Action: Export summary
    Action-->>User: Return results

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• New action implementation: Verify parameter mapping, date parsing logic, and export structure align with Trawlingweb API expectations
• Client refactoring: Ensure _makeRequest() properly injects authentication and that searchNews() delegation preserves existing behavior
• Dependency integration: Confirm @pipedream/platform is the correct import source for axios and that version constraint is appropriate

Poem

🐰 Hop along the trawling way,
News retrieved throughout the day,
Requests bundled, tokens tight,
Search and sort with newfound might!
Version bumped, the flow takes flight! ✨

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The repository specifies a required template with a "WHY" section for pull request descriptions. The author-provided description only contains "Resolves #10972" and does not follow the template structure by filling in or explaining the "WHY" section. While the description does reference the linked issue, it fails to comply with the required template format and lacks substantive explanation of the purpose behind these changes.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title "Trawlingweb - new component" is concise, clear, and directly describes the primary change in the pull request. It accurately summarizes that a new Trawlingweb component is being added to the codebase. The title is specific and avoids vague terminology, making it immediately understandable to teammates reviewing the commit history.
Out of Scope Changes Check	✅ Passed	All changes in the pull request directly support the implementation of the Trawlingweb component. The modifications include the new search-news action file, updates to the Trawlingweb app module to support the action, and necessary package.json updates for dependencies and versioning. No unrelated or out-of-scope changes are present in the changeset.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch issue-10972

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 36a8917 and 859cc8b.

⛔ Files ignored due to path filters (1)

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (3)

• components/trawlingweb/actions/search-news/search-news.mjs (1 hunks)
• components/trawlingweb/package.json (2 hunks)
• components/trawlingweb/trawlingweb.app.mjs (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

• GitHub Check: Publish TypeScript components
• GitHub Check: Verify TypeScript components
• GitHub Check: pnpm publish
• GitHub Check: Lint Code Base

🔇 Additional comments (8)

components/trawlingweb/package.json (1)

3-3: LGTM!

The version bump to 0.1.0 is appropriate for the new action functionality, and the @pipedream/platform dependency is correctly added to support the axios usage in the app module.

Also applies to: 14-17

components/trawlingweb/trawlingweb.app.mjs (3)

1-1: LGTM!

The axios import from @pipedream/platform is correctly added to support the HTTP request functionality.

---

23-27: LGTM!

The delegation pattern is clean and allows flexibility by passing through all options to the underlying request method.

---

8-22: The code implementation is correct for the Trawlingweb API.

The Trawlingweb API expects requests to the base URL https://api.trawlingweb.com with search terms passed as query parameters (token for API key and q for the query). The code correctly constructs this by setting the url to the base URL and passing token and other parameters through the params object, which axios will convert to query parameters in the request.

Likely an incorrect or invalid review comment.

components/trawlingweb/actions/search-news/search-news.mjs (4)

1-13: LGTM!

The action metadata and structure follow Pipedream conventions correctly, with appropriate annotations for a read-only search operation.

---

14-60: LGTM!

The props are well-defined with clear labels, descriptions, and appropriate type constraints. The max: 100 validation on maxResults is a good safeguard.

---

62-62: Verify API response structure.

The code assumes the axios response has the shape { response: { data } }. Please confirm this matches the actual Trawlingweb API response structure. The optional chaining on line 80 (data?.length) is good defensive coding.

Also applies to: 80-82

---

66-74: Verify date format and sort parameter logic against official API documentation.

The code contains a discrepancy that requires verification:

1. Date format conflict: Props describe dateStart and dateEnd as "ISO 8601 format (YYYY-MM-DD)", but the code converts these strings to millisecond timestamps using Date.parse(). Verify against the official Trawlingweb API documentation whether the ts and tsi parameters expect ISO date strings or millisecond timestamps.

2. Sort parameter ambiguity: The code sets sort: "published" only when selected, otherwise undefined. Confirm whether sending undefined (or omitting the parameter) for the "crawled" sort option is the intended API behavior, or if the code should pass this.sort directly.

Without access to the official API documentation, these behaviors cannot be confirmed as correct. Please verify the expected parameter formats in the Trawlingweb dashboard and update the code accordingly.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[GTFalcao]

LGTM!