scrapy 命令行常用命令大全

scrapy 命令行常用命令大全

scrapy startproject myproject

#myproject为项目名

scrapy genspider mydomain mydomain.com

#mydomain为spider文件名，mydomain.com为爬取网站域名

全局命令：

startproject

genspider

settings

runspider

shell

fetch

view

version

项目内部命令（局部命令）：

crawl

check

list

edit

parse

bench

scrapy crawl <spider>

scrapy crawl <spider> --nolog

scrapy check

scrapy list

scrapy edit <spider>

scrapy fetch <url>

scrapy view <url>

scrapy shell [url]

scrapy parse <url> [options]

scrapy settings [options]

scrapy settings --get BOT_NAME

scrapy runspider <spider_file.py>

scrapy version [-v]

scrapy bench #测试电脑当前爬取速度性能