- You can do things like setting a download delay between each request, limiting amount of concurrent requests per domain or per IP, and even using an auto-throttling extension that tries to figure out these automatically. link
scrapy_doc_tips
原文：https://www.cnblogs.com/jyh-py-blog/p/10797176.html
