Git-Mirrors/censorbib
1
0
Fork 0
mirror of https://github.com/NullHypothesis/censorbib.git synced 2025-07-23 14:41:07 -04:00
Code Issues Projects Releases Packages Wiki Activity

Use the latest Chrome user agent to bypass NDSS's HTTP 403.

Browse source
This commit is contained in:
gfw-report 2022-03-26 05:21:57 +00:00
parent dd95fc5ca8
commit d749b3b314
No known key found for this signature in database
GPG key ID: B0C6EB19DA7CEAA3
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
fetch_pdfs.py
View file

@ -34,7 +34,8 @@ def download_pdf(url, file_name):
print("Now fetching %s" % url)
try:
fetched_file = urllib.request.urlopen(url)
req = urllib.request.Request(url, headers={'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36"})
fetched_file = urllib.request.urlopen(req)
except Exception as err:
print(url, err, file=sys.stderr)
return