Skip to content

Options, proper finishing by crawling strategy and traceback on SIGUSR1

Choose a tag to compare

View all tags
@sibiryakov sibiryakov released this 29 Jun 08:09
· 529 commits to master since this release
v0.5.2.2
62d9e49
  • CONSUMER_BATCH_SIZE is removed and two new options are introduced SPIDER_LOG_CONSUMER_BATCH_SIZE and SCORING_LOG_CONSUMER_BATCH_SIZE
  • Traceback is thrown into log when SIGUSR1 is received in DBW or SW.
  • Finishing in SW is fixed when crawling strategy reports finished.
Assets 2
Loading