python爬虫程序报错:ConnectionResetError
程序如下,爬取过程中会报错,报错时间随机,有时候爬几百条就报错,有时候爬几千条报错:`
ConnectionResetError: [Errno 54] Connection reset by peer
def get_page(url):
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36"
}
reqeust = urllib.request.Request(url=url, headers=headers)
try:
response = urllib.request.urlopen(reqeust)
page = response.read().decode('utf-8')
# print(html)
except urllib.error.URLError as e:
if hasattr(e, "code"):
print(e.code)
if hasattr(e, "reason"):
print(e.reason)
return page
File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 33, in get_data
page = get_page(url)
File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 21, in get_page
page = response.read().decode('utf-8')
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 471, in read
s = self._safe_read(self.length)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 612, in _safe_read
data = self.fp.read(amt)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
回答:
网络不稳定很正常的现象,你可以加个重试
for i in range(3):
response = urllib.request.urlopen(reqeust)
以上是 python爬虫程序报错:ConnectionResetError 的全部内容, 来源链接: utcz.com/a/161013.html