SCP10: Incomplete project throttling
What it does
Reports a setting module (e.g. settings.py) where
CONCURRENT_REQUESTS_PER_DOMAIN and DOWNLOAD_DELAY are not
both set.
Why is this bad?
A Scrapy project should set sane defaults for throttling settings based on the kind of websites that it aims to scrape.
Additional, more granular adjustments can be made on a per-domain basis
(DOWNLOAD_SLOTS) or on a per-spider basis (e.g. in
custom_settings).
Example
CONCURRENT_REQUESTS_PER_DOMAIN = 1
DOWNLOAD_DELAY = 1.0