Python爬虫入门-爬取新浪新闻

运行环境：Python3.6.0

所需的包：

from bs4 import BeautifulSoup
import requests

response = requests.get("http://news.sina.com.cn/china/")
response.encoding = "utf-8"
soup = BeautifulSoup(response.text, "lxml")
headers = soup.select("div.news-item > h2")
links = soup.select("div.news-item > h2 > a")
times = soup.select("div.time")

for header, link, time in zip(headers, links, times):
    with open("sina_news.txt", "a") as f:
        f.write(header.get_text() + "\n" +
                time.get_text() + "\n" +
                link.get("href") +
                "\n---------------------\n")

爬取结果：

Python爬虫入门-爬取新浪新闻.JPG