SCP04: Improper response selector

What it does

Finds usage of Selector constructed with TextResponse objects or response text/body that can be replaced with response selector methods.

Why is this bad?

Creating a new Selector instance when you already have a TextResponse object is unnecessary and less efficient. Scrapy’s TextResponse object provides convenient methods like xpath(), css(), and selector that are more direct and readable.

These built-in methods also handle encoding properly and are optimized for common use cases.

Example

import scrapy
from parsel import Selector


class MySpider(scrapy.Spider):
    name = "myspider"

    def parse(self, response):
        selector = Selector(response)
        yield {"title": selector.css("h1::text").get()}

Use instead:

import scrapy


class MySpider(scrapy.Spider):
    name = "myspider"

    def parse(self, response):
        yield {"title": response.css("h1::text").get()}