Is it possible to run a crawler forever?

[galaczi]

I am wondering if it's possible to run a crawler forever? I cannot set infinity on requestHandlerTimeoutSecs and I think setting it to zero also timed out at the default timeout.

[B4nan]

In latest, version you can use keepAlive: true in crawler options, that will keep the crawler running even if the request queue is empty. You can shut it down via crawler.teardown() afterwards.

[galaczi]

Thanks! I am also hoping requestHandlerTimeoutSecs: 0 will be available at some point to completely remove timeout from handler.