SCP02: URL in allowed_domains

What it does

Finds URLs in allowed_domains instead of domain names.

Why is this bad?

The allowed_domains attribute should contain domain names only, not full URLs.

Example

import scrapy


class MySpider(scrapy.Spider):
    name = "myspider"
    allowed_domains = ["https://toscrape.com/"]

Use instead:

import scrapy


class MySpider(scrapy.Spider):
    name = "myspider"
    allowed_domains = ["toscrape.com"]