Skip to content

Commit 59ee9a3

Browse files
committed
feat: improvements
1 parent 7c7aec9 commit 59ee9a3

File tree

2 files changed

+38
-9
lines changed

2 files changed

+38
-9
lines changed

sources/platform/integrations/workflows-and-notifications/n8n/ai-crawling.md

Lines changed: 38 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,19 +7,19 @@ slug: /integrations/n8n/ai-crawling
77
toc_max_heading_level: 4
88
---
99

10+
## Apify Scraper for AI Crawling
11+
12+
Apify Scraper for AI Crawling from [Apify](https://console.apify.com/actors/aYG0l9s7dbB7j3gbS/input) lets you extract text content from websites to feed AI models, LLM applications, vector databases, or Retrieval Augmented Generation (RAG) pipelines. It supports rich formatting using Markdown, cleans the HTML of irrelevant elements, downloads linked files, and integrates with AI ecosystems like LangChain, LlamaIndex, and other LLM frameworks.
13+
14+
To use these modules, you need an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate content extraction at scale and incorporate the results into your AI workflows.
15+
1016
## Prerequisites
1117

1218
Before you begin, make sure you have:
1319

1420
- An [Apify account](https://console.apify.com/)
1521
- An [n8n instance](https://docs.n8n.io/getting-started/) (self‑hosted or cloud)
1622

17-
## Apify Scraper for AI Crawling
18-
19-
Apify Scraper for AI Crawling from [Apify](https://apify.com/) lets you extract text content from websites to feed AI models, LLM applications, vector databases, or Retrieval Augmented Generation (RAG) pipelines. It supports rich formatting using Markdown, cleans the HTML of irrelevant elements, downloads linked files, and integrates with AI ecosystems like LangChain, LlamaIndex, and other LLM frameworks.
20-
21-
To use these modules, you need an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate content extraction at scale and incorporate the results into your AI workflows.
22-
2323
## Install the Apify Node (self-hosted)
2424

2525
If you're running a self-hosted n8n instance, you can install the Apify community node directly from the editor. This process adds the node to your available tools, enabling Apify operations in workflows.
@@ -33,6 +33,20 @@ If you're running a self-hosted n8n instance, you can install the Apify communit
3333

3434
![Apify Install Node](images/install.png)
3535

36+
## Install the Apify Scraper for AI Crawling Node (n8n Cloud)
37+
38+
For n8n Cloud users, installation is even simpler and doesn't require manual package entry. Just search and add the node from the canvas.
39+
40+
1. Go to the **Canvas** and open the **nodes panel**
41+
1. Search for **Apify Scraper for AI Crawling** in the community node registry
42+
1. Click **Install node** to add the Apify node to your instance
43+
44+
:::note Verified community nodes visibility
45+
46+
On n8n Cloud, instance owners can toggle visibility of verified community nodes in the Cloud Admin Panel. Ensure this setting is enabled to install the Apify Scraper for AI Crawling node.
47+
48+
:::
49+
3650
## Connect Apify Scraper for AI Crawling (self-hosted)
3751

3852
1. Create an account at [Apify](https://console.apify.com/). You can sign up using your email, Gmail, or GitHub account.
@@ -53,9 +67,24 @@ If you're running a self-hosted n8n instance, you can install the Apify communit
5367

5468
![Apify token on n8n](images/token.png)
5569

56-
Once connected, you can build workflows to automate website extraction and integrate results into your AI applications.
70+
### OAuth2 (cloud instance only)
71+
72+
1. In n8n Cloud, select **Create Credential**.
73+
1. Search for Apify OAuth2 API and select **Continue**.
74+
1. Select **Connect my account** and authorize with your Apify account.
75+
1. n8n automatically retrieves and stores the OAuth2 tokens.
76+
77+
![Apify Auth](../../images/n8n-oauth.png)
78+
79+
:::note
80+
81+
For simplicity on n8n Cloud, use the API key method if you prefer manual control over credentials.
82+
83+
:::
84+
85+
With authentication set up, you can now create workflows that incorporate the Apify node.
5786

58-
## Apify Scraper for Website Content modules
87+
## Apify Scraper for AI Crawling modules
5988

6089
After connecting the app, you can use one of the two modules as native scrapers to extract website content.
6190

@@ -136,7 +165,7 @@ You can access any of thousands of our scrapers on Apify Store by using the [gen
136165

137166
## Usage as an AI Agent Tool
138167

139-
You can setup Apify's Website Content Crawler app as a tool for your AI Agents.
168+
You can setup Apify's Scraper for AI Crawling node as a tool for your AI Agents.
140169

141170
![Setup AI Agent](./images/setup.png)
142171

-67 KB
Loading

0 commit comments

Comments
 (0)