Skip to content

Commit 471f3f8

Browse files
MQ37MQ37
authored
feat: add crewai integration docs (#1484)
Integration docs for crewAIInc/crewAI-tools#228 and crewAIInc/crewAI#2254. Will be merged after these PRs are merged.
1 parent 99e55b5 commit 471f3f8

File tree

1 file changed

+209
-0
lines changed

1 file changed

+209
-0
lines changed

‎sources/platform/integrations/ai/crewai.md

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
---
2+
title: 🤖🚀 CrewAI integration
3+
sidebar_label: CrewAI
4+
description: Learn how to build AI Agents with Apify and CrewAI 🤖🚀.
5+
sidebar_position: 1
6+
slug: /integrations/crewai
7+
---
8+
9+
**Learn how to build AI Agents with Apify and CrewAI.**
10+
11+
---
12+
13+
## What is CrewAI
14+
15+
[CrewAI](https://www.crewai.com/) is an open-source Python framework designed to orchestrate autonomous, role-playing AI agents that collaborate as a "crew" to tackle complex tasks. It enables developers to define agents with specific roles, assign tasks, and integrate tools—like Apify Actors—for real-world data retrieval and automation.
16+
17+
:::note Explore CrewAI
18+
19+
For more in-depth details on CrewAI, check out its [official documentation](https://docs.crewai.com/).
20+
21+
:::
22+
23+
## How to use Apify with CrewAI
24+
25+
This guide demonstrates how to integrate Apify Actors with CrewAI by building a crew of agents that uses the [RAG Web Browser](https://apify.com/apify/rag-web-browser) Actor to search Google for TikTok profiles and the [TikTok Data Extractor](https://apify.com/clockworks/free-tiktok-scraper) Actor to extract and analyze data from the TikTok profiles.
26+
27+
### Prerequisites
28+
29+
- **Apify API token**: To use Apify Actors in CrewAI, you need an Apify API token. Learn how to obtain it in the [Apify documentation](https://docs.apify.com/platform/integrations/api).
30+
- **OpenAI API key**: To power the agents in CrewAI, you need an OpenAI API key. Get one from the [OpenAI platform](https://platform.openai.com/account/api-keys).
31+
- **Python packages**: Install the following Python packages:
32+
33+
```bash
34+
pip install 'crewai[tools]' langchain-apify langchain-openai
35+
```
36+
37+
### Building the TikTok profile search and analysis crew
38+
39+
First, import all required packages:
40+
41+
```python
42+
import os
43+
from crewai import Agent, Task, Crew
44+
from crewai_tools import ApifyActorsTool
45+
from langchain_openai import ChatOpenAI
46+
```
47+
48+
Next, set the environment variables for the Apify API token and OpenAI API key:
49+
50+
```python
51+
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"
52+
os.environ["APIFY_API_TOKEN"] = "Your Apify API token"
53+
```
54+
55+
Instantiate the LLM and Apify Actors tools:
56+
57+
```python
58+
llm = ChatOpenAI(model="gpt-4o-mini")
59+
60+
browser_tool = ApifyActorsTool(actor_name="apify/rag-web-browser")
61+
tiktok_tool = ApifyActorsTool(actor_name="clockworks/free-tiktok-scraper")
62+
```
63+
64+
Define the agents with roles, goals, and tools:
65+
66+
```python
67+
search_agent = Agent(
68+
role="Web Search Specialist",
69+
goal="Find the TikTok profile URL on the web",
70+
backstory="Expert in web searching and data retrieval",
71+
tools=[browser_tool],
72+
llm=llm,
73+
verbose=True
74+
)
75+
76+
analysis_agent = Agent(
77+
role="TikTok Profile Analyst",
78+
goal="Extract and analyze data from the TikTok profile",
79+
backstory="Skilled in social media data extraction and analysis",
80+
tools=[tiktok_tool],
81+
llm=llm,
82+
verbose=True
83+
)
84+
```
85+
86+
Define the tasks for the agents:
87+
88+
```python
89+
search_task = Task(
90+
description="Search the web for the OpenAI TikTok profile URL.",
91+
agent=search_agent,
92+
expected_output="A URL linking to the OpenAI TikTok profile."
93+
)
94+
95+
analysis_task = Task(
96+
description="Extract data from the OpenAI TikTok profile URL and provide a profile summary and details about the latest post.",
97+
agent=analysis_agent,
98+
context=[search_task],
99+
expected_output="A summary of the OpenAI TikTok profile including followers and likes, plus details about their most recent post."
100+
)
101+
```
102+
103+
Create and run the crew:
104+
105+
```python
106+
crew = Crew(
107+
agents=[search_agent, analysis_agent],
108+
tasks=[search_task, analysis_task],
109+
process="sequential"
110+
)
111+
112+
result = crew.kickoff()
113+
print(result)
114+
```
115+
116+
:::note Search and analysis may take some time
117+
118+
The agent tasks may take some time as they search the web for the OpenAI TikTok profile and extract data from it.
119+
120+
:::
121+
122+
You will see the crew’s output in the console, showing the results of the search and analysis.
123+
124+
```text
125+
Profile Summary:
126+
- Username: OpenAI
127+
- Profile URL: [OpenAI TikTok Profile](https://www.tiktok.com/@openai)
128+
- Followers: 605,000
129+
- Likes: 3,400,000
130+
- Number of Videos: 152
131+
- Verified: Yes
132+
- Signature: low key research previews
133+
- Bio Link: [OpenAI Website](https://openai.com/)
134+
135+
Latest Post Details:
136+
- Post ID: 7474019216346287406
137+
- Post Text: "@Adeline Mai is a photographer..."
138+
- Creation Time: February 21, 2025
139+
- Number of Likes: 863
140+
- Number of Shares: 26
141+
- Number of Comments: 33
142+
- Number of Plays: 20,400
143+
- Number of Collects: 88
144+
- Music Used: Original Sound by OpenAI
145+
- Web Video URL: [Watch Here](https://www.tiktok.com/@openai/video/7474019216346287406)
146+
```
147+
148+
If you want to test the whole example, create a new file, `crewai_integration.py`, and copy the full code into it:
149+
150+
```python
151+
import os
152+
from crewai import Agent, Task, Crew
153+
from crewai_tools import ApifyActorsTool
154+
from langchain_openai import ChatOpenAI
155+
156+
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"
157+
os.environ["APIFY_API_TOKEN"] = "Your Apify API token"
158+
159+
llm = ChatOpenAI(model="gpt-4o-mini")
160+
161+
browser_tool = ApifyActorsTool(actor_name="apify/rag-web-browser")
162+
tiktok_tool = ApifyActorsTool(actor_name="clockworks/free-tiktok-scraper")
163+
164+
search_agent = Agent(
165+
role="Web Search Specialist",
166+
goal="Find the TikTok profile URL on the web",
167+
backstory="Expert in web searching and data retrieval",
168+
tools=[browser_tool],
169+
llm=llm,
170+
verbose=True
171+
)
172+
173+
analysis_agent = Agent(
174+
role="TikTok Profile Analyst",
175+
goal="Extract and analyze data from the TikTok profile",
176+
backstory="Skilled in social media data extraction and analysis",
177+
tools=[tiktok_tool],
178+
llm=llm,
179+
verbose=True
180+
)
181+
182+
search_task = Task(
183+
description="Search the web for the OpenAI TikTok profile URL.",
184+
agent=search_agent,
185+
expected_output="A URL linking to the OpenAI TikTok profile."
186+
)
187+
analysis_task = Task(
188+
description="Extract data from the OpenAI TikTok profile URL and provide a profile summary and details about the latest post.",
189+
agent=analysis_agent,
190+
context=[search_task],
191+
expected_output="A summary of the OpenAI TikTok profile including followers and likes, plus details about their most recent post."
192+
)
193+
194+
crew = Crew(
195+
agents=[search_agent, analysis_agent],
196+
tasks=[search_task, analysis_task],
197+
process="sequential"
198+
)
199+
200+
result = crew.kickoff()
201+
print(result)
202+
```
203+
204+
## Resources
205+
206+
- [Apify Actors](https://docs.apify.com/platform/actors)
207+
- [CrewAI Documentation](https://docs.crewai.com/)
208+
- [What are AI agents?](https://blog.apify.com/what-are-ai-agents/)
209+
- [How to build an AI agent](https://blog.apify.com/how-to-build-an-ai-agent/)

0 commit comments

Comments
 (0)