没有内置设置可以做到这一点，但是这仍然可以很容易地实现。
在我看来，最直接的方法是在你的spider中捕捉response_received信号，并在你的ISP故障时检查你收到的特定错误代码。当这种情况发生时，您可以暂停scrapy引擎并等待任意时间，然后再次重试相同的请求，直到成功。
例如：

from scrapy import Spider
from scrapy.signals import response_received

class MySpider(Spider):
   ...
   ...

    @classmethod
    def from_crawler(cls, crawler, *args,**kwargs):
        spider = super(MySpider, cls).from_crawler(crawler, *args,**kwargs)
        # listen for the response_received signal and call check_response
        crawler.signals.connect(spider.check_response, signal=response_received)
        return spider

    def check_response(self, response, request, spider):
        engine = spider.crawler.engine
        if response.status == 404:  # <- your error code goes here
            engine.pause()
            time.sleep(6000)        # <- wait 10 minutes
            request.dont_filter = True   # <- tell engine not to filter
            engine.unpause()
            engine.crawl(request.copy())  # <- resend the request

使现代化
因为它不是http错误代码，所以下一个最好的解决方案是创建一个自定义DownloaderMiddleware来捕获异常，然后执行与第一个示例中相同的操作。
在middlewares.py文件中：

import time
from twisted.internet.error import (ConnectError, ConnectionLost
                                    TimeoutError, DNSLookupError,
                                    ConnectionRefusedError)

class ConnectionLostPauseDownloadMiddleware:

    def __init__(self, settings, crawler):
        self.crawler = crawler
        self.exceptions = (ConnectionRefusedError, ConnectionDone, ConnectError, ConnectionLost)

    @classmethod
    def from_crawler(cls, crawler):
        return cls(crawler.settings, crawler)

    def process_exception(self, request, exception, spider):
        if isinstance(exception, self.exceptions):
            new_request = request.copy()
            new_request.dont_filter = True
            self.crawler.engine.pause()
            time.sleep(60 * 10)
            self.crawler.engine.unpause()
            return new_request

然后在你的settings.py中

DOWNLOADER_MIDDLEWARES = {
   'MyProjectName.middlewares.ConnectionLostPauseDownloadMiddleware': 543,
}