DataForSEO - new components #18601
DataForSEO - new components #18601
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
WalkthroughAdds two new DataForSEO OnPage actions for creating crawl tasks and retrieving crawled pages, updates an existing OnPage parse action’s metadata, and bumps the package version and a dependency. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor U as User/Workflow
participant A as Action: Create OnPage Task
participant DFS as DataForSEO OnPage API
U->>A: Trigger run()
A->>DFS: POST /on_page/task_post {target, max_crawl_pages, ...}
DFS-->>A: Response {status_code, tasks[0].status_code, ...}
alt Valid status codes (20000/20100)
A-->>U: Summary: Successfully created onpage task.
else Error status
A-->>U: Throw ConfigurationError with status_message
end
sequenceDiagram
autonumber
actor U as User/Workflow
participant A as Action: Get Crawled Pages
participant DFS as DataForSEO OnPage API
U->>A: Trigger run()
A->>DFS: POST /on_page/pages {id, limit, search_after_token, tag}
DFS-->>A: Response {status_code, tasks[0].status_code, ...}
alt Valid status codes (20000)
A-->>U: Summary: Successfully retrieved crawled pages.
else Error status
A-->>U: Throw ConfigurationError with status_message
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
components/dataforseo/actions/create-onpage-task/create-onpage-task.mjs (1)
23-27: Consider adding validation for maxCrawlPages.Adding a
minconstraint would prevent users from submitting invalid values (e.g., 0 or negative numbers).maxCrawlPages: { type: "integer", label: "Max Crawl Pages", description: "The number of pages to crawl on the specified domain", + min: 1, },
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (4)
components/dataforseo/actions/create-onpage-task/create-onpage-task.mjs(1 hunks)components/dataforseo/actions/get-crawled-pages/get-crawled-pages.mjs(1 hunks)components/dataforseo/actions/parse-page-content/parse-page-content.mjs(1 hunks)components/dataforseo/package.json(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
components/dataforseo/actions/get-crawled-pages/get-crawled-pages.mjs (2)
components/dataforseo/actions/parse-page-content/parse-page-content.mjs (1)
response(61-72)components/dataforseo/actions/create-onpage-task/create-onpage-task.mjs (1)
response(69-82)
components/dataforseo/actions/create-onpage-task/create-onpage-task.mjs (2)
components/dataforseo/actions/parse-page-content/parse-page-content.mjs (1)
response(61-72)components/dataforseo/actions/get-crawled-pages/get-crawled-pages.mjs (1)
response(51-61)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Verify TypeScript components
- GitHub Check: Publish TypeScript components
- GitHub Check: pnpm publish
- GitHub Check: Lint Code Base
🔇 Additional comments (9)
components/dataforseo/package.json (1)
3-3: LGTM! Version and dependency updates are appropriate.The version bump to 0.4.0 correctly reflects the addition of new OnPage actions, and the platform dependency upgrade to ^3.1.0 aligns with the latest stable release.
Also applies to: 16-16
components/dataforseo/actions/parse-page-content/parse-page-content.mjs (1)
6-6: LGTM! Metadata updates align with OnPage API focus.The name clarification and version bump appropriately reflect the OnPage API integration theme of this PR.
Also applies to: 9-9
components/dataforseo/actions/get-crawled-pages/get-crawled-pages.mjs (3)
1-14: LGTM! Imports and metadata are well-configured.The ConfigurationError import and action metadata follow Pipedream conventions. The
readOnlyHint: trueannotation correctly reflects that this action retrieves data without modifying resources.
41-49: LGTM! Method implementation follows established patterns.The
getCrawledPagehelper correctly delegates todataforseo._makeRequestand matches the pattern used in other DataForSEO actions.
22-27: ConfirmedlimitandtagpropDefinitions exist indataforseo.app.mjs; no further action needed.components/dataforseo/actions/create-onpage-task/create-onpage-task.mjs (4)
1-14: LGTM! Imports and metadata are correctly configured.The
readOnlyHint: falseannotation appropriately reflects that this action creates a new task.
59-67: LGTM! Method implementation is consistent.The helper method follows the established pattern used across DataForSEO actions.
68-82: LGTM! API request payload is correctly structured.The property mapping from camelCase props to snake_case API fields follows DataForSEO API conventions.
17-22: target propDefinition verifiedThe
targetpropDefinition exists in the DataForSeo app’s definitions, so no further action is required.
Resolves #18503
Summary by CodeRabbit
New Features
Style
Chores