Skip to content

Commit c514252

Browse files
authored
docs: restructure Academy (#2040)
As per slack discussion & notion, here is PoC with reworked structure & placeholders for new content. Work done so far: - [x] create new Build and publish section - [x] move content - [x] move images - [x] remove H1's - [x] fix formatting - [x] create placeholders - [x] remove AI section Work to be done: - [x] fix initial Vale issues - [x] remove placeholder descriptions from pages - [x] add new content - [x] Why publish Actors - [x] What software an Actor can be - [x] Validation framework - [x] Affiliates
1 parent c945082 commit c514252

File tree

65 files changed

+633
-183
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+633
-183
lines changed

.github/styles/config/vocabularies/Docs/accept.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ SKUs?
118118
subreddits?
119119
[Ss]ubwoofer
120120
Tripadvisor
121-
upvote
121+
[Uu]pvote
122122
[Ww]alkthroughs?
123123

124124
ul
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
label: How to pick an Actor idea
2+
position: 2
Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
---
2+
title: Validate your Actor idea
3+
description: Learn how to validate market demand for your Actor using SEO data, community research, and competitive analysis before you build.
4+
sidebar_position: 2
5+
category: build-and-publish
6+
slug: /build-and-publish/actor-ideas/actor-validation
7+
---
8+
9+
Before investing time into building an Actor, validate that people actually need it. This guide shows you how to assess market demand using free tools and research techniques.
10+
11+
## Assess your motivation
12+
13+
Ask yourself: _Do you want to build this?_
14+
15+
You'll work on this Actor for a long time. The best Actors come from developers who genuinely care about the problem they're solving. You don't need to be obsessed, but you should feel excited. That enthusiasm carries you through challenges and shows
16+
in your work.
17+
18+
## Estimate demand with SEO data
19+
20+
Check if people are searching for solutions like yours. If your idea aligns with popular search queries, you have a built-in user base.
21+
22+
### Keyword demand
23+
24+
Search for terms related to your Actor's function. If you're building a Reddit sentiment analysis scraper, check volume for phrases like _Reddit data extractor_ or _analyze Reddit comments tool_.
25+
26+
Use free tools:
27+
28+
- [Google Keyword Planner](https://business.google.com/en-all/ad-tools/keyword-planner/)
29+
- [Whatsmyserp](https://chromewebstore.google.com/detail/whatsmyserp/chbmoagfhnkggnhbjpoonnmhnpjdjdod) Chrome extension
30+
- [Keywords Everywhere](https://keywordseverywhere.com/) (paid)
31+
32+
High search volume or multiple related terms indicate solid demand. Low or zero searches mean a very niche market, which isn't bad, but you'll rely more on direct marketing.
33+
34+
### Google autocomplete and related searches
35+
36+
Type your core keywords into Google and note the suggestions. Typing _scrape Amazon_ might show _scrape Amazon reviews_ or _Amazon price tracker_, confirming what people actually want.
37+
38+
### SEO difficulty and content gaps
39+
40+
Examine current search results. Few quality results for a query like _download data from [obscure site]_ indicates a content gap your Actor can fill.
41+
42+
Many results or ads for _Instagram scraper_ means the market is proven but competitive. You'll need to differentiate.
43+
44+
Check keyword difficulty and domain authority. If difficulty is 70+ and top pages have 80+ domain authority with thousands of backlinks—and Apify already has an official Actor with 100,000+ users—you can't compete directly. Find an adjacent angle or specialization.
45+
46+
## Analyze Google Trends
47+
48+
Google Trends shows if interest in your idea is rising or falling. Declining trends are red flags. If searches dropped 90% over 12+ months (like _Clubhouse scraper_ since 2021), that market has moved on.
49+
50+
Growth velocity matters more than current volume. A keyword growing from 10 to 100 monthly searches over 12 months shows exploding demand. Jump in early, before competition heats up.
51+
52+
Watch for spikes. Sudden jumps from media coverage or viral moments usually don't mean sustainable demand.
53+
54+
## Research community discussions
55+
56+
Beyond SEO data, go where your potential users are. Browse Reddit, Hacker News, Stack Overflow, X (Twitter), Discord, and Facebook groups. What problems are people discussing? What tools do they wish existed?
57+
58+
Document your findings. Note quotes and recurring themes like _Multiple marketers on Reddit want easy competitor pricing tracking—no existing solution mentioned_. These insights complement your SEO data and help you speak your users' language.
59+
60+
Zero discussion across multiple platforms over 4+ weeks means either no one cares about the problem or they've already solved it.
61+
62+
### Reddit
63+
64+
Search relevant subreddits (r/webscraping, r/datascience, r/SEO, r/marketing, or industry-specific ones) for questions like _How can I extract [data] from [site]?_ or _I wish there was a tool to do X_. Multiple people independently asking for the same solution is strong validation.
65+
66+
Use the `site:` parameter in Google to search for relevant threads:
67+
68+
```text
69+
70+
site:reddit.com extracting data from LinkedIn
71+
72+
```
73+
74+
You can also use tools like [F5Bot](https://f5bot.com/) or
75+
[GummySearch](https://gummysearch.com/).
76+
77+
### Q&A forums and Stack Overflow
78+
79+
Look for questions about doing the task manually. If thinking about a LinkedIn scraper, check Stack Overflow for questions like _How can I scrape LinkedIn profiles?_ Frequent questions or upvotes indicate many people trying to solve it without a good tool—an opportunity for your Actor.
80+
81+
Use the `site:` parameter:
82+
83+
```text
84+
85+
site:stackoverflow.com extracting data from LinkedIn
86+
87+
```
88+
89+
### X and social media
90+
91+
Search keywords on X, LinkedIn, or other social media for professionals asking for recommendations like _Does anyone know a tool to monitor news about [topic]?_
92+
93+
Run quick polls or ask your followers if they'd use a tool that does XYZ. A few positive responses validate your idea. Silence means rethink your value proposition. Engaging this way is early marketing.
94+
95+
Use the `site:` parameter:
96+
97+
```text
98+
99+
site:x.com extracting data from LinkedIn
100+
101+
```
102+
103+
### Hacker News and niche forums
104+
105+
Platforms like [Hacker News](https://news.ycombinator.com/) often have discussions on tech pain points and new tool launches. Search for keywords like _scrape Airbnb data_ to see if people have shown interest or if someone launched a similar tool and what the reaction was.
106+
107+
Use the `site:` parameter:
108+
109+
```text
110+
111+
site:news.ycombinator.com extracting data from LinkedIn
112+
113+
```
114+
115+
:::tip Look for spending signals
116+
117+
Current spending patterns are the strongest signal. When users mention "currently paying $X/month for [existing tool] but..." or "upgraded from free to paid plan because..." or specific competitor pricing, they are proven buyers.
118+
119+
You can also engage in communities. Answer related questions, share knowledge, build reputation. Mention your Actor idea casually where relevant: "I'm building a tool to solve exactly this, would you use it?" Track responses. Positive responses with questions about pricing or features mean genuine interest.
120+
121+
:::
122+
123+
## Analyze GitHub repositories
124+
125+
Star counts signal market demand. [Scrapy](https://github.com/scrapy/scrapy) has 58,000+, [Crawlee](https://github.com/apify/crawlee) has 20,000+, web scraping is validated. Use the [Star History tool](https://www.star-history.com/) to check if stars are rising (growing momentum) or flat.
126+
127+
Issue analysis reveals pain points your Actor could solve. High issue counts with active responses indicate healthy, used projects. Open issues with themes like _JavaScript rendering problems_ or _CAPTCHA bypass needed_ show gaps you can fill. Issues with 10+ upvotes mean multiple users face the same problem.
128+
129+
Fork and commit activity shows developers actively work with the technology. High fork-to-star ratios mean people are building extensions (evidence of real usage). Recent commits (within 30 days) indicate active maintenance and a healthy project. No commits for 6+ months suggests declining interest.
130+
131+
## Review Product Hunt launches
132+
133+
Study successful automation tool launches from the past 12-24 months on Product Hunt. Filter by _Browser Automation_ and _Automation tools_, then sort by upvotes. Note which taglines, value propositions, and features resonated. Products with 500+ upvotes validated something—figure out what worked.
134+
135+
## Research Apify Store
136+
137+
Apify Store shows transparent competitive intelligence most marketplaces hide. Every Actor displays monthly users, ratings, pricing, and last updates, a data goldmine for what works and what doesn't.
138+
139+
Search your use case or segment thoroughly. List relevant Actors with their metrics: monthly users, ratings, pricing, last update, and creator. Create a feature comparison matrix. Analyze top performers' READMEs, documentation quality, and issues.
140+
141+
Review competitor issues tabs closely. High-quality READMEs with examples and clear value propositions perform better in Store search. Issues reveal unresolved pain points from actual users. If competitors have 20+ open issues with repeated themes, that's your differentiation roadmap.
142+
143+
### Assess market saturation
144+
145+
- 10-30 Actors: healthy competition (market validated, you need differentiation)
146+
- 50+ Actors: saturated (need obvious gaps)
147+
- 1-5 Actors: blue ocean or unproven demand (validate carefully)
148+
149+
If the market has 50+ Actors with strong leaders (Apify-maintained with 50,000+ users) and you can't articulate clear differentiation, pivot. If you spot feature gaps or underserved niches, continue.
150+
151+
## Scan the broader market
152+
153+
Do a general Google search for tools or services that solve your problem. Your competition might not be another Actor—it could be a SaaS tool or API. If your idea is _monitor website uptime and screenshot changes_, established services probably exist.
154+
155+
Note direct competitors: How do they price it? What audience do they target? Are users satisfied or complaining? This validates that people pay for the service and reveals gaps you can fill.
156+
157+
Understanding the competition helps you refine your unique value—whether that's lower cost, better features, or targeting an underserved niche.
158+
159+
No existing solutions? Ask why. You might have found an untapped need, or it's a red flag (too difficult to implement, or the target website aggressively blocks scraping). Use your judgment.
160+
161+
## Get feedback from potential users
162+
163+
Reach out to people who match your target user profile. Building a real estate data Actor? Contact real estate analysts or agents (LinkedIn works well) and ask if a tool that does X would help them. Keep it informal—describe the problem you're solving and ask if they'd use or pay for it.
164+
165+
Direct feedback helps you:
166+
167+
- Validate your assumptions
168+
- Understand pricing expectations
169+
- Identify must-have features
170+
- Refine your value proposition
171+
172+
Track responses carefully. Enthusiasm with specific questions about features or pricing indicates genuine interest. Generic "sounds interesting" responses mean keep validating.
Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: Find ideas for your Actor
3+
description: Discover what types of software you can build as Actors and use Apify's Ideas page to find trending projects for scrapers, integrations, and automation tools.
4+
sidebar_position: 1
5+
category: build-and-publish
6+
slug: /build-and-publish/actor-ideas/find-actor-ideas
7+
---
8+
9+
You want to build an Actor and publish it on Apify Store. Where should you start? See what types of software work as Actors and how to use the Apify Ideas page to find projects that users need.
10+
11+
---
12+
13+
## What can you build as an Actor
14+
15+
Actors are serverless cloud applications that run on the Apify platform. While most Actors are web scrapers, you can build various types of software as Actors can be anything that accepts input, performs a job, and runs in Docker.
16+
17+
### Web scrapers and crawlers
18+
19+
The most common type of Actor extracts data from websites. These Actors navigate web pages, collect information, and store structured data in datasets.
20+
21+
Examples:
22+
23+
- Site-specific scrapers ([Amazon Product Scraper](https://apify.com/junglee/amazon-crawler), [LinkedIn Profile Scraper](https://apify.com/curious_coder/linkedin-profile-scraper))
24+
- Search engine scrapers ([Google Search Results](https://apify.com/apify/google-search-scraper), [Bing Search](https://apify.com/curious_coder/bing-search-scraper))
25+
- Social media scrapers ([X/Twitter Scraper](https://apify.com/apidojo/twitter-scraper-lite), [Instagram Scraper](https://apify.com/apify/instagram-scraper))
26+
- E-commerce data extractors ([Shopify Store Scraper](https://apify.com/autofacts/shopify), [eBay Price Monitor](https://apify.com/dtrungtin/ebay-items-scraper))
27+
28+
### SaaS API wrappers
29+
30+
Convert existing SaaS services into Actors to make them accessible through the Apify platform. This approach lets users integrate these services with Apify's automation ecosystem.
31+
32+
Example:
33+
34+
- [Parsera](https://apify.com/parsera-labs/parsera) wraps the Parsera.org service
35+
36+
### Open-source libraries
37+
38+
Package open-source tools as Actors to provide cloud-hosted versions that users can try without local installation. You handle no infrastructure while users cover compute costs.
39+
40+
Examples:
41+
42+
- [Monolith](https://apify.com/snshn/monolith) from the Y2Z/monolith GitHub project
43+
- [Crawl4AI](https://apify.com/janbuchar/crawl4ai) from the unclecode/crawl4ai repository
44+
- [Docling](https://apify.com/vancura/docling/source-code) from IBM's
45+
docling-project
46+
47+
:::tip Open Source Fair Share
48+
49+
Open-source developers can earn income through [Apify's Open Source Fair Share program](https://apify.com/partners/open-source-fair-share).
50+
51+
:::
52+
53+
### MCP servers
54+
55+
Model Context Protocol (MCP) servers let AI agents interact with external tools and data sources. Converting MCP servers to Actors makes them accessible through Apify's platform.
56+
57+
Examples:
58+
59+
- [Playwright MCP](https://apify.com/jiri.spilka/playwright-mcp-server) from Microsoft's playwright-mcp
60+
- [Browserbase MCP](https://apify.com/mcp-servers/browserbase-mcp-server) from Browserbase
61+
62+
### AI agents
63+
64+
Build Actors that use LLMs to perform complex tasks autonomously. These Actors can navigate websites, make decisions, and complete multi-step workflows.
65+
66+
## Use the Ideas page
67+
68+
The [Apify Ideas page](https://apify.com/ideas) is where users submit and explore potential projects for Actors. It serves as a collaborative space for proposing new tools and finding inspiration for web scraping and automation solutions.
69+
70+
### Browse and claim ideas
71+
72+
If you're unsure what to build next, the Ideas page shows projects the community wants.
73+
74+
Steps to develop an idea:
75+
76+
1. Browse the page: Visit [apify.com/ideas](https://apify.com/ideas) to find ideas that interest you. Look for ideas that align with your skills.
77+
78+
1. Select an idea: Review the details and requirements. Check the status—if it's marked **Open to develop**, you can start building.
79+
80+
1. Build your Actor: Develop your Actor based on the idea. You don't need to notify Apify during development.
81+
82+
1. Prepare for launch: Ensure your Actor meets quality standards and has a comprehensive README with installation instructions, usage details, and examples.
83+
84+
1. Publish your Actor: Deploy your Actor on Apify Store and make it live.
85+
86+
1. Claim your idea: After publishing, email [[email protected]](mailto:[email protected]) with your Actor URL and the original idea. Apify will tag the idea as _Completed_ and link it to your Actor.
87+
88+
1. Monitor and optimize: Track your Actor's performance and user feedback. Make improvements to keep your Actor current.
89+
90+
### Criteria for claiming an idea
91+
92+
To claim an idea, ensure:
93+
94+
- Your Actor is functional
95+
- Your README contains relevant information
96+
- Your Actor closely aligns with the original idea
97+
98+
### Submit your own ideas
99+
100+
The Ideas page is also where you contribute concepts to drive innovation in the community.
101+
102+
How to contribute:
103+
104+
1. Submit ideas: Share Actor concepts through the [Ideas submission
105+
form](https://apify.typeform.com/to/BNON8poB#source=ideas). Provide clear details about what the tool should do and how it should work.
106+
107+
1. Engage with the community: Upvote ideas you find intriguing. More support
108+
increases the likelihood a developer will build it.
109+
110+
1. Claim completed ideas: Once your Actor is running, claim your idea by emailing [[email protected]](mailto:[email protected]). Apify will mark it
111+
_Completed_ and link it to your Actor—signaling to other developers that this tool exists.
112+
113+
### Multiple developers for one idea
114+
115+
Apify Store can host multiple Actors with similar functions. However, the "first come, first served" rule applies—the first developer to claim an idea receives the _Completed_ tag and a link from the Ideas page.
116+
117+
Competition helps developers improve their code. You can still build the Actor, but differentiate with a unique set of features.
118+
119+
## Generate ideas from other sources
120+
121+
Beyond the Ideas page, you can find Actor ideas through:
122+
123+
- SEO tools: Discover relevant search terms people use to find solutions
124+
- Your experience: Draw from problems you've encountered in your work
125+
- Community discussions: Browse Reddit, Stack Overflow, and forums for pain
126+
points
127+
- Competitor analysis: Research existing tools and identify gaps
128+
129+
For a complete validation framework, see [Validate your Actor idea](/academy/build-and-publish/actor-ideas/actor-validation).
Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,20 @@
11
---
22
title: Actor success stories
33
description: Learn about developers who successfully make passive income from their Actors.
4-
sidebar_position: 5
5-
category: apify platform
4+
sidebar_position: 7
5+
category: build-and-publish
66
slug: /actor-marketing-playbook/store-basics/actor-success-stories
77
---
88

99
_Web scraping freelance financial freedom with microworlds._
1010

11-
Discover how Caleb David, founder of `microworlds`, achieved financial freedom through freelance web scraping. His journey showcases how mastering the craft with tools like Crawlee and creating a Twitter scraper transformed his career. See the full story [here](https://blog.apify.com/web-scraping-freelance-financial-freedom/) and learn from his success.
11+
Discover how Caleb David, founder of `microworlds`, achieved financial freedom through freelance web scraping. His journey showcases how mastering the craft with tools like Crawlee and creating a Twitter scraper transformed his career. See the full story in this [blog post](https://blog.apify.com/web-scraping-freelance-financial-freedom/) and learn from his success.
1212

1313
https://apify.com/microworlds
1414

1515
_Web scraping for freelance success – insights from Tugkan._
1616

17-
In this success story, our first community dev Tugkan shares how his journey into freelancing via Apify changed his life. Learn about his process, challenges, and how his paid Actors have brought him financial rewards and freedom. Check out his story [here](https://apify.com/success-stories/paid-actor-journey-apify-freelancer-tugkan) for inspiration.
17+
In this success story, our first community dev Tugkan shares how his journey into freelancing via Apify changed his life. Learn about his process, challenges, and how his paid Actors have brought him financial rewards and freedom. Check out his [story](https://apify.com/success-stories/paid-actor-journey-apify-freelancer-tugkan) for inspiration.
1818

1919
https://apify.com/epctex
2020

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
label: Apify Store basics
2+
position: 4
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Actor description & SEO description
33
description: Learn about Actor description and meta description. Where to set them and best practices for both content and length.
44
sidebar_position: 3
5-
category: apify platform
5+
category: build-and-publish
66
slug: /actor-marketing-playbook/actor-basics/actor-description
77
---
88

@@ -70,7 +70,7 @@ Setting the SEO description and SEO name is optional. If not set, the descriptio
7070

7171
![changing actor name and seo name](images/changing_Actor_name_and_SEO_name.png)
7272

73-
Actor description specifically can also be quick-edited in this pop-up on the Actor's page in Apify Console. Open the **Actor's page**, then click on **** in the top right corner, and choose ✎ **Edit name or description**. Then set the URL in the **Unique name** ✎ field and click **Save**.
73+
Actor description specifically can also be quick-edited in this pop-up on the Actor's page in Apify Console. Open the **Actor's page**, then click on **** in the top right corner, and choose ✎ **Edit name or description**. Then set the URL in the **Unique name** ✎ field and click **Save**.
7474

7575
![changing actor description](images/change_Actor_description.png)
7676

sources/academy/platform/get_most_of_actors/actor_basics/actors-and-emojis.md renamed to sources/academy/build-and-publish/apify-store-basics/actors_and_emojis.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
2-
title: Actors & emojis
2+
title: Actors and emojis
33
description: Discover how emojis can boost your Actors by grabbing attention, simplifying navigation, and enhancing clarity. Improve user experience and engagement on Apify Store.
4-
sidebar_position: 5
5-
category: apify platform
4+
sidebar_position: 4
5+
category: build-and-publish
66
slug: /actor-marketing-playbook/actor-basics/actors-and-emojis
77
---
88

0 commit comments

Comments
 (0)