headers如何设置Referer

2024-10-12 01:33:09

1、获取网页源代码及设置http请求头 1 import requests、2 from bs4 import BeautifulSoup、3 import os。4headers = {'User-Agent':"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 " "(KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1"}.

3、解析后获取对应的标签并创建文件夹all_a = Soup.find('div',class_='all').findAll('a')for a in all_a

5、获取单个页面中的源码，并解析，同12步骤类似

6、headers = {'User-Agent':"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWe芟鲠阻缒bKit/537.1 " "(KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1" ,'Referer'。