爬虫中的模拟登录

使用Fiddler 抓取网络数据，登录'https://www.v2ex.com/'，抓到一堆包，使用find功能标记出'v2ex'的有用包，看它的cookies：

Paste_Image.png

可以发现最关键的那个HTTP会话包里面，
Request Count: 1
Bytes Sent: 1,185 (headers:1,185; body:0)
Bytes Received: 84,209 (headers:632; body:83,577)

点击Inspectors，查看核心参数。

代码如下：

import requests
from bs4 import BeautifulSoup
 
url = "http://www.v2ex.com/signin"
UA = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.13 Safari/537.36"
 
header = { "User-Agent" : UA,
           "Referer": "http://www.v2ex.com/signin"
           }
 
v2ex_session = requests.Session()
f = v2ex_session.get(url,headers=header)
 
soup = BeautifulSoup(f.content,"html.parser")
once = soup.find('input',{'name':'once'})['value']
print(once)
 
postData = { 'u': 'whatbeg',
             'p': '*****',
             'once': once,
             'next': '/'
             }
 
v2ex_session.post(url,
                  data = postData,
                  headers = header)
 
f = v2ex_session.get('http://www.v2ex.com/settings',headers=header)
print(f.content.decode())

这个网站早已改版了：
不过经过RSA加密后的数据也可以在Fiddler里面抓出来，
第一次请求的TextView 里面可以看到数据：
dea6a48e5539a63cb15c7ee309fbdf422a27b31a52a52a92ce188e7bca63de9e=your name&38fe18af1d70cea04c1f16c06106c1c529f65240f4e0ebaeb98ed02ba20026e4=your password&once=57443&next=%2F

dea6a48e5539a63cb15c7ee309fbdf422a27b31a52a52a92ce188e7bca63de9e是用户名的加密值
38fe18af1d70cea04c1f16c06106c1c529f65240f4e0ebaeb98ed02ba20026e4是密码的加密值
once是明文

代码如下：

import requests
import re
import json
from bs4 import BeautifulSoup
 
s = requests.Session()
 
headers = {
    'Accept': 'application/json, text/javascript, */*; q=0.01',
    'Referer': 'http://passport.cnblogs.com/user/signin',
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36',
    'Cookie': '__gads=ID=fc58354935efbd89:T=1458638388:S=ALNI_MYEtsucyem4nWeL9mdxvQmfAZlTgQ; _ga=GA1.2.111229817.1458781632; .CNBlogsCookie=39EB7C846FF5A6CA5D762D210B954E55CE77A24D11C5203F6055DCAC93DFFF8EA7E405568F2D8CC9F00AFE43A859E71DE55AE6E79A030F7E74C231CECF7DA2DD88B734EA2ECA22DFED8C2ECAB85717B45434AABFE1202DA8266C7440562114D99D9C6767'
}
 
login_data = {'input1': '你的用户名加密后内容',
              'input2': '你的密码加密后内容',
              'remember': 'false'
              }
 
url = 'http://passport.cnblogs.com/user/signin'
req = s.post(url, data = login_data, headers=headers)
print(req.status_code)           #200
print(req.content.decode())      #{"success":false,"message":"您已处于登录状态"}
 
f = s.get('http://home.cnblogs.com/u/whatbeg/followers/1', headers=headers)
print(f.status_code)
print(f.text)