Skip to content

Severe degradation of performance #2087

@bghtrbb

Description

@bghtrbb

Which package is this bug report for? If unsure which one to select, leave blank

None

Issue description

Here I have 2 performance screen shots a 1 week and 1 hour closeup that is running in the cloud. As you can see there is a severe decline in performance the longer it run.

Explanation for the flat section....
The flat part you see at the end of the graph is from when the container kept shutting down due to issue #2085. Then the line ends when I shut it off.

But once I fixed that you can see the performance continues at the same poor performance. So the 1 hour close up is from the last hour after I got it back and running. 5+ minute stops.

At this point there are around 190,000 results.

1 Week
CrawleePerformace1week

1 Hour Closeup
CrawleePerformace1hour

Code sample

No response

Package version

3.5.3

Node.js version

18.16.0

Operating system

Linux On ECS Fargate

Apify platform

  • Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions