Skip to content

Upgrade Django task for lastest TWRH crawler SP: 5 #191

@ddio

Description

@ddio
  1. Use list crawler w/ headless browser enabled
  2. Only trigger detail crawler when:
    1. It's a new item,
    2. it's opened, but not listed in list crawler (from DB instead)
    3. or if it's outdated, hasn't been updated for over 1 month
  3. Handle sync house status w/o daily table to reduce disk usage

New fields introduced (TBD):

  1. opened_at
  2. closed_at

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions