Skip to content

scraper takes too long to run #6

@mrcnc

Description

@mrcnc

Currently, on my machine with a 2.2 GHz i7 Processor, it's crawling ~600 pages/minute. Since there are ~165000 parcels, it will take ~4.5 hours to complete. This is too long and it would be a lot better if we could speed up this process.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions