Skip to content

Add the script that can repeatedly scrapes the coursebook until finished #108

@mikehquan19

Description

@mikehquan19

Currently we have -resume flag for coursebook scraper that can resume scraping the coursebook until it scrapes everything. Check it out if you haven't yet.
To take a step further in automating the coursebook scraper in GCP, add the shell script that will repeatedly call the coursebook scraper with resume flag (in case it is terminted halfway) until every course is scraped successfully.

Make sure to put the limit on the number of attemps it can recall the resume.

Metadata

Metadata

Labels

Help WantedExtra attention is neededL2A task suitable for someone who is comfortable helping with implementing features.Type: Feature RequestNew feature or request

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions