python爬虫程序报错:ConnectionResetError

python爬虫程序报错:ConnectionResetError

程序如下,爬取过程中会报错,报错时间随机,有时候爬几百条就报错,有时候爬几千条报错:`
ConnectionResetError: [Errno 54] Connection reset by peer

def get_page(url):

headers = {

"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36"

}

reqeust = urllib.request.Request(url=url, headers=headers)

try:

response = urllib.request.urlopen(reqeust)

page = response.read().decode('utf-8')

# print(html)

except urllib.error.URLError as e:

if hasattr(e, "code"):

print(e.code)

if hasattr(e, "reason"):

print(e.reason)

return page

File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 33, in get_data

page = get_page(url)

File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 21, in get_page

page = response.read().decode('utf-8')

File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 471, in read

s = self._safe_read(self.length)

File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 612, in _safe_read

data = self.fp.read(amt)

File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto

return self._sock.recv_into(b)


回答:

网络不稳定很正常的现象,你可以加个重试
for i in range(3):
response = urllib.request.urlopen(reqeust)

以上是 python爬虫程序报错:ConnectionResetError 的全部内容, 来源链接: utcz.com/a/161013.html

回到顶部