Git-Mirrors/real-world-onion-sites
1
0
Fork 0
mirror of https://github.com/alecmuffett/real-world-onion-sites.git synced 2025-01-11 07:09:27 -05:00
Code Issues Packages Projects Releases Wiki Activity

commit: retry_limit is now 6, nits

Browse Source
This commit is contained in:
Alec Muffett 2019-11-16 21:34:36 +00:00
parent f5a3695cfb
commit a935df67f2
3 changed files with 2 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@ -280,13 +280,6 @@ This is a list of substantial, commercial-or-social-good mainstream websites whi
* :white_check_mark: attempt=1 code=200 time=2019-11-16 18:00:34+00:00 * :white_check_mark: attempt=1 code=200 time=2019-11-16 18:00:34+00:00
* :white_check_mark: attempt=1 code=200 time=2019-11-16 16:45:16+00:00 * :white_check_mark: attempt=1 code=200 time=2019-11-16 16:45:16+00:00
### [Whonix Wiki](http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Main_page)
* [http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Main_page](http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Main_page) :wrench:
* proof: [link](https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/)
* :negative_squared_cross_mark: attempt=1 code=404 time=2019-11-16 18:44:36+00:00
* :negative_squared_cross_mark: attempt=1 code=404 time=2019-11-16 18:00:36+00:00
* :negative_squared_cross_mark: attempt=1 code=404 time=2019-11-16 16:45:18+00:00
### [keybase.io](http://keybase5wmilwokqirssclfnsqrjdsi7jdir5wy7y7iu3tanwmtp6oid.onion/) ### [keybase.io](http://keybase5wmilwokqirssclfnsqrjdsi7jdir5wy7y7iu3tanwmtp6oid.onion/)
* [http://keybase5wmilwokqirssclfnsqrjdsi7jdir5wy7y7iu3tanwmtp6oid.onion/](http://keybase5wmilwokqirssclfnsqrjdsi7jdir5wy7y7iu3tanwmtp6oid.onion/) :wrench: * [http://keybase5wmilwokqirssclfnsqrjdsi7jdir5wy7y7iu3tanwmtp6oid.onion/](http://keybase5wmilwokqirssclfnsqrjdsi7jdir5wy7y7iu3tanwmtp6oid.onion/) :wrench:
* proof: [link](https://keybase.io/docs/command_line/tor) * proof: [link](https://keybase.io/docs/command_line/tor)

1
master.csv
View File

@ -109,7 +109,6 @@ tech and software,Tor Project Onion Index,http://yz7lpwfhhzcdyc5y.onion/,,https:
tech and software,Whonix,http://kkkkkkkkkk63ava6.onion/,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/, tech and software,Whonix,http://kkkkkkkkkk63ava6.onion/,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/,
tech and software,Whonix Forums,http://forums.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/, tech and software,Whonix Forums,http://forums.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/,
tech and software,Whonix Home,http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/, tech and software,Whonix Home,http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/,
tech and software,Whonix Wiki,http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Main_page,,https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/,
web and internet,Archive Today (archive.is),http://archivecaslytosk.onion/,,https://archive.is/, web and internet,Archive Today (archive.is),http://archivecaslytosk.onion/,,https://archive.is/,
web and internet,Cloudflare Public DNS 1.1.1.1,https://dns4torpnlfs2ifuz2s2yf3fc7rdmsbhm6rw75euj35pac6ap25zgqad.onion/,,ssl, web and internet,Cloudflare Public DNS 1.1.1.1,https://dns4torpnlfs2ifuz2s2yf3fc7rdmsbhm6rw75euj35pac6ap25zgqad.onion/,,ssl,
web and internet,DuckDuckGo,https://3g2upl4pq6kufc4m.onion/,,ssl, web and internet,DuckDuckGo,https://3g2upl4pq6kufc4m.onion/,,ssl,

1 category site_name onion_url flaky proof_url comment
109 tech and software Whonix http://kkkkkkkkkk63ava6.onion/ https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/
110 tech and software Whonix Forums http://forums.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/ https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/
111 tech and software Whonix Home http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/ https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/
tech and software Whonix Wiki http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Main_page https://www.qubes-os.org/news/2018/01/23/qubes-whonix-next-gen-tor-onion-services/
112 web and internet Archive Today (archive.is) http://archivecaslytosk.onion/ https://archive.is/
113 web and internet Cloudflare Public DNS 1.1.1.1 https://dns4torpnlfs2ifuz2s2yf3fc7rdmsbhm6rw75euj35pac6ap25zgqad.onion/ ssl
114 web and internet DuckDuckGo https://3g2upl4pq6kufc4m.onion/ ssl

3
rwos-db.py
View File

@ -17,6 +17,7 @@ USER_AGENT = "Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0"
BADNESS = 900 BADNESS = 900
CURL_TIMEOUT = 120 CURL_TIMEOUT = 120
RETRY_SLEEP = 60 RETRY_SLEEP = 60
RETRY_LIMIT = 6
PLACEHOLDER = '-' PLACEHOLDER = '-'
POOL_WORKERS = 10 POOL_WORKERS = 10
YES = 'y' YES = 'y'
@ -158,7 +159,7 @@ class URL:
)) ))
def fetchwrap(self): def fetchwrap(self):
for i in range(5): for i in range(RETRY_LIMIT):
self.fetch1() self.fetch1()
print('try{0}: {1} {2}'.format(i, self.url, self.last_code)) print('try{0}: {1} {2}'.format(i, self.url, self.last_code))
if self.last_code < BADNESS: return if self.last_code < BADNESS: return