You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+28-2Lines changed: 28 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -131,6 +131,7 @@ x-crawl is an open source project under the MIT license, completely free to use.
131
131
-[AnyObject](#anyobject)
132
132
-[FAQ](#faq)
133
133
-[The relationship between crawlPage API and puppeteer](#the-relationship-between-crawlpage-api-and-puppeteer)
134
+
-[Using crawlPage API causes the program to crash](#using-crawlpage-api-causes-the-program-to-crash)
134
135
-[More](#more)
135
136
-[Community](#community)
136
137
-[Issues](#issues)
@@ -334,8 +335,6 @@ Lifecycle functions owned by the crawlPage API:
334
335
335
336
In the onCrawlItemComplete function, you can get the results of each crawled goal in advance.
336
337
337
-
**Note:** If you need to crawl many pages at one time, you need to use this life cycle function to process the results of each target and close the page instance after each page is crawled down. If you do not close the page instance, then The program will crash due to too many opened pages.
The crawlPage API has built-in [puppeteer](https://github.com/puppeteer/puppeteer), you only need to pass in some configuration options to let x-crawl help you simplify the operation and get the intact Brower instance and Page instance , x-crawl does not override it.
2074
2073
2074
+
### Using crawlPage API causes the program to crash
2075
+
2076
+
If you need to crawl many pages in one crawlPage, it is recommended that after crawling each page, use [onCrawlItemComplete life cycle function] (#onCrawlItemComplete) to process the results of each target and close the page instance. If no shutdown operation is performed, then The program may crash due to too many pages being opened (related to the performance of the device itself).
Copy file name to clipboardExpand all lines: publish/README.md
+28-2Lines changed: 28 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -131,6 +131,7 @@ x-crawl is an open source project under the MIT license, completely free to use.
131
131
-[AnyObject](#anyobject)
132
132
-[FAQ](#faq)
133
133
-[The relationship between crawlPage API and puppeteer](#the-relationship-between-crawlpage-api-and-puppeteer)
134
+
-[Using crawlPage API causes the program to crash](#using-crawlpage-api-causes-the-program-to-crash)
134
135
-[More](#more)
135
136
-[Community](#community)
136
137
-[Issues](#issues)
@@ -334,8 +335,6 @@ Lifecycle functions owned by the crawlPage API:
334
335
335
336
In the onCrawlItemComplete function, you can get the results of each crawled goal in advance.
336
337
337
-
**Note:** If you need to crawl many pages at one time, you need to use this life cycle function to process the results of each target and close the page instance after each page is crawled down. If you do not close the page instance, then The program will crash due to too many opened pages.
The crawlPage API has built-in [puppeteer](https://github.com/puppeteer/puppeteer), you only need to pass in some configuration options to let x-crawl help you simplify the operation and get the intact Brower instance and Page instance , x-crawl does not override it.
2074
2073
2074
+
### Using crawlPage API causes the program to crash
2075
+
2076
+
If you need to crawl many pages in one crawlPage, it is recommended that after crawling each page, use [onCrawlItemComplete life cycle function] (#onCrawlItemComplete) to process the results of each target and close the page instance. If no shutdown operation is performed, then The program may crash due to too many pages being opened (related to the performance of the device itself).
0 commit comments