|
| 1 | +--- |
| 2 | +title: Make - LLMs Actor integration |
| 3 | +description: Learn about LLM browser modules. Search the web and extract clean Markdown for AI assistants and RAG. |
| 4 | +sidebar_label: LLMs |
| 5 | +sidebar_position: 7 |
| 6 | +slug: /integrations/make/llm |
| 7 | +toc_max_heading_level: 4 |
| 8 | +--- |
| 9 | + |
| 10 | +## Apify Scraper for LLMs |
| 11 | + |
| 12 | +Apify Scraper for LLMs from [Apify](https://apify.com) is a web browsing module for OpenAI Assistants, RAG pipelines, and AI agents. It can query Google Search, scrape the top results, and return page content as Markdown for downstream AI processing. |
| 13 | + |
| 14 | +To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the Apify Console under **Settings > Integrations**. After connecting, you can automate content extraction and integrate results into your AI workflows. |
| 15 | + |
| 16 | +## Connect Apify Scraper for LLMs |
| 17 | + |
| 18 | +1. Create an account at [Apify](https://console.apify.com/). You can sign up using your email, Gmail, or GitHub account. |
| 19 | + |
| 20 | +  |
| 21 | + |
| 22 | +1. To connect your Apify account to Make, you can use an OAuth connection (recommended) or an Apify API token. To get the token, go to **[Settings > API & Integrations](https://console.apify.com/settings/integrations)** in the Apify Console. |
| 23 | + |
| 24 | +  |
| 25 | + |
| 26 | +1. Find your token under **Personal API tokens**. You can also create a new token with custom permissions by clicking **+ Create a new token**. |
| 27 | +1. Click the **Copy** icon to copy your API token, then return to your Make scenario. |
| 28 | + |
| 29 | +  |
| 30 | + |
| 31 | +1. In Make, click **Add** to open the **Create a connection** dialog of the chosen Apify Scraper module. |
| 32 | +1. In the **API token** field, paste your token, provide a clear **Connection name**, and click **Save**. |
| 33 | + |
| 34 | +  |
| 35 | + |
| 36 | +Once connected, you can build workflows that search the web, extract content, and pass it to your AI applications. |
| 37 | + |
| 38 | +## Apify Scraper for LLMs modules |
| 39 | + |
| 40 | +After connecting the app, you can use two modules to search and extract content. |
| 41 | + |
| 42 | +### Standard Settings module |
| 43 | + |
| 44 | +Use Standard Settings to quickly search the web and extract content with optimized defaults. This is ideal for AI agents that need to answer questions or gather information from multiple sources. |
| 45 | + |
| 46 | +The module supports two modes: |
| 47 | + |
| 48 | +- _Search mode_ (keywords) |
| 49 | + - Queries Google Search with your keywords (supports advanced operators) |
| 50 | + - Retrieves the top N organic results |
| 51 | + - Loads each result and extracts the main content |
| 52 | + - Returns Markdown-formatted content |
| 53 | + |
| 54 | +- _Direct URL mode_ (URL) |
| 55 | + - Navigates to a specific URL |
| 56 | + - Extracts page content |
| 57 | + - Skips Google Search |
| 58 | + |
| 59 | +#### How it works |
| 60 | + |
| 61 | +When you provide keywords, the module runs Google Search, parses the results, and collects organic URLs. For content extraction, it loads pages, waits for dynamic content to render, removes clutter, extracts the main content, and converts it to Markdown. Finally, it generates output by combining content, adding metadata and sources, and formatting everything for AI consumption. |
| 62 | + |
| 63 | +#### Output data |
| 64 | + |
| 65 | +```json title="Standard Settings output (shortened)" |
| 66 | +{ |
| 67 | + "query": "web browser for RAG pipelines -site:reddit.com", |
| 68 | + "crawl": { |
| 69 | + "httpStatusCode": 200, |
| 70 | + "httpStatusMessage": "OK", |
| 71 | + "loadedAt": "2025-06-30T10:15:23.456Z", |
| 72 | + "uniqueKey": "https://example.com/article", |
| 73 | + "requestStatus": "handled" |
| 74 | + }, |
| 75 | + "searchResult": { |
| 76 | + "title": "Building RAG Pipelines with Web Browsers", |
| 77 | + "description": "Integrate web browsing into your RAG pipeline for real-time retrieval.", |
| 78 | + "url": "https://example.com/article", |
| 79 | + "resultType": "organic", |
| 80 | + "rank": 1 |
| 81 | + }, |
| 82 | + "metadata": { |
| 83 | + "title": "Building RAG Pipelines with Web Browsers", |
| 84 | + "description": "Add web browsing to RAG systems", |
| 85 | + "languageCode": "en", |
| 86 | + "url": "https://example.com/article" |
| 87 | + }, |
| 88 | + "markdown": "# Building RAG Pipelines with Web Browsers\n\n..." |
| 89 | +} |
| 90 | +``` |
| 91 | + |
| 92 | +#### Configuration (Standard Settings) |
| 93 | + |
| 94 | +- _Search query_: Google Search keywords or a direct URL |
| 95 | +- _Maximum results_: Number of top search results to process (default: 3) |
| 96 | +- _Output formats_: Markdown, text, or HTML |
| 97 | +- _Remove cookie warnings_: Dismiss cookie consent dialogs |
| 98 | +- _Debug mode_: Enable extraction diagnostics |
| 99 | + |
| 100 | +### Advanced Settings module |
| 101 | + |
| 102 | +Advanced Settings give you full control over search and extraction. Use it for complex sites or production RAG pipelines. |
| 103 | + |
| 104 | +#### Key features |
| 105 | + |
| 106 | +- _Advanced search options_: full Google operator support |
| 107 | +- _Flexible crawling tools_: browser-based (Playwright) or HTTP-based (Cheerio) |
| 108 | +- _Proxy configuration_: handle geo-restrictions and rate limits |
| 109 | +- _Granular content control_: include, remove, and click selectors |
| 110 | +- _Dynamic content handling_: wait strategies for JavaScript rendering |
| 111 | +- _Multiple output formats_: Markdown, HTML, or text |
| 112 | +- _Request management_: timeouts, retries, and concurrency |
| 113 | + |
| 114 | +#### Configuration options |
| 115 | + |
| 116 | +- _Search_: query, max results (1–100), SERP proxy group, SERP retries |
| 117 | +- _Scraping_: tool (browser-playwright, raw-http), HTML transformer, selectors (remove/keep/click), expand clickable elements |
| 118 | +- _Requests_: timeouts, retries, dynamic content wait |
| 119 | +- _Proxy_: use Apify Proxy, proxy groups, countries |
| 120 | +- _Output_: formats, save HTML/Markdown, debug mode, save screenshots |
| 121 | + |
| 122 | +#### Output data |
| 123 | + |
| 124 | +```json title="Advanced Settings output (shortened)" |
| 125 | +{ |
| 126 | + "query": "advanced RAG implementation strategies", |
| 127 | + "crawl": { |
| 128 | + "httpStatusCode": 200, |
| 129 | + "httpStatusMessage": "OK", |
| 130 | + "loadedUrl": "https://ai-research.com/rag-strategies", |
| 131 | + "loadedTime": "2025-06-30T10:45:12.789Z", |
| 132 | + "referrerUrl": "https://www.google.com/search?q=advanced+RAG+implementation+strategies", |
| 133 | + "uniqueKey": "https://ai-research.com/rag-strategies", |
| 134 | + "requestStatus": "handled", |
| 135 | + "depth": 0 |
| 136 | + }, |
| 137 | + "searchResult": { |
| 138 | + "title": "Advanced RAG Implementation: A Complete Guide", |
| 139 | + "description": "Cutting-edge strategies for RAG systems.", |
| 140 | + "url": "https://ai-research.com/rag-strategies", |
| 141 | + "resultType": "organic", |
| 142 | + "rank": 1 |
| 143 | + }, |
| 144 | + "metadata": { |
| 145 | + "canonicalUrl": "https://ai-research.com/rag-strategies", |
| 146 | + "title": "Advanced RAG Implementation: A Complete Guide | AI Research", |
| 147 | + "description": "Vector DBs, chunking, and optimization techniques.", |
| 148 | + "languageCode": "en" |
| 149 | + }, |
| 150 | + "markdown": "# Advanced RAG Implementation: A Complete Guide\n\n...", |
| 151 | + "debug": { |
| 152 | + "extractorUsed": "readableText", |
| 153 | + "elementsRemoved": 47, |
| 154 | + "elementsClicked": 3 |
| 155 | + } |
| 156 | +} |
| 157 | +``` |
| 158 | + |
| 159 | +### Use cases |
| 160 | + |
| 161 | +- Quick information retrieval for AI assistants |
| 162 | +- General web search integration and Q&A |
| 163 | +- Production RAG pipelines that need reliability |
| 164 | +- Extracting content from JavaScript-heavy sites |
| 165 | +- Building specialized knowledge bases and research workflows |
| 166 | + |
| 167 | +### Best practices |
| 168 | + |
| 169 | +To get the best search results, use specific keywords and operators, and exclude unwanted domains with `-site:`. For better performance, use HTTP mode for static sites and only switch to browser mode when necessary. You can also tune concurrency settings based on your needs. To maintain content quality, remove non-content elements, choose the right HTML transformer, and enable debug mode when troubleshooting. Finally, ensure reliable operation by setting appropriate timeouts and retries, and monitoring HTTP status codes for errors. |
| 170 | + |
| 171 | +## Other scrapers available |
| 172 | + |
| 173 | +There are other native Make Apps powered by Apify. You can check out Apify Scraper for: |
| 174 | + |
| 175 | +- [TikTok Data](/platform/integrations/make/tiktok) |
| 176 | +- [Google Search](/platform/integrations/make/search) |
| 177 | +- [Google Maps Emails Data](/platform/integrations/make/maps) |
| 178 | +- [YouTube Data](/platform/integrations/make/youtube) |
| 179 | +- [AI crawling](/platform/integrations/make/ai-crawling) |
| 180 | +- [Amazon](/platform/integrations/make/amazon) |
| 181 | + |
| 182 | +And more! Because you can access any of thousands of our scrapers on Apify Store by using the [general Apify connections](https://www.make.com/en/integrations/apify). |
0 commit comments