CentOS7+Python3环境安装Scrapy

关于CentOS7里Python3的安装详见《CentOS 7 安装 Python3.5》

步骤：

1：安装Twisted

因为Scrapy需要用到Twisted支持所以需要先安装Twisted，Twisted点这里去官网下载。

wget https://pypi.python.org/packages/31/bf/7f86a8f8b9778e90d8b2921e9f442a8c8aa33fd2489fc10f236bc8af1749/Twisted-17.5.0.tar.bz2#md5=cd5c287802dcbaf7be15cf937c922b71

tar jxvf Twisted-17.5.0.tar.bz2 #解压缩

cd Twisted-17.5.0

python setup.py install

注意：

tar解压bz2时返回错误：

Child returned status 2

解决办法：

yum install bzip2 #tar直接解压bz2格式会出错，所以需要安装bzip2支持

2：安装Scrapy

mkdir scrapy

cd scrapy

source bin/activate

(scrapy)pip install scrapy

(scrapy)scrapy 

如果返回：

Scrapy 1.4.0 - no active projectUsage:  scrapy[options] [args]Available commands:

  bench        Run quick benchmark test

  fetch        Fetch a URL using the Scrapy downloader

  genspider    Generate new spider using pre-defined templates

  runspider    Run a self-contained spider (without creating a project)

  settings      Get settings values

  shell        Interactive scraping console

  startproject  Create new project

  version      Print Scrapy version

  view          Open URL in browser, as seen by Scrapy

  [ more ]      More commands available when run from project directoryUse "scrapy-h" to see more info about a command

则表示成功，此外版本为：1.4.0