Git-Mirrors/real-world-onion-sites
1
0
Fork 0
mirror of https://github.com/alecmuffett/real-world-onion-sites.git synced 2024-10-01 01:06:18 -04:00
Code Issues Packages Projects Releases Wiki Activity

auto-update on Sun Apr 10 11:24:49 UTC 2022

Browse Source
This commit is contained in:
Alec Muffett 2022-04-10 11:24:49 +00:00
parent fee0557d28
commit fc542971fc
3 changed files with 11 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
02-footnotes.md
View File

@ -17,6 +17,7 @@
- :sos: site returned no data, or is down, or curl experienced a transient or permanent network error; may also reflect a problem with the RWOS server connection - :sos: site returned no data, or is down, or curl experienced a transient or permanent network error; may also reflect a problem with the RWOS server connection
- :boom: same as :sos: but curl specifically mentioned SSL certificates as an issue - :boom: same as :sos: but curl specifically mentioned SSL certificates as an issue
- :alarm_clock: same as :sos: but curl specifically mentioned timeout as an issue - :alarm_clock: same as :sos: but curl specifically mentioned timeout as an issue
- :question: same as :sos: but curl specifically mentioned inability to resolve an onion descriptor
- :new: site is newly added, no data yet - :new: site is newly added, no data yet
You can also see the [history of updates](https://github.com/alecmuffett/real-world-onion-sites/commits/master/README.md). You can also see the [history of updates](https://github.com/alecmuffett/real-world-onion-sites/commits/master/README.md).

5
README.md
View File

@ -83,7 +83,7 @@ You can find techical details and the legend/key for symbols in the [footnotes s
* link: [https://xw226dvxac7jzcpsf4xb64r4epr6o5hgn46dxlqk7gnjptakik6xnzqd.onion/](https://xw226dvxac7jzcpsf4xb64r4epr6o5hgn46dxlqk7gnjptakik6xnzqd.onion/) * link: [https://xw226dvxac7jzcpsf4xb64r4epr6o5hgn46dxlqk7gnjptakik6xnzqd.onion/](https://xw226dvxac7jzcpsf4xb64r4epr6o5hgn46dxlqk7gnjptakik6xnzqd.onion/)
* plain: `https://xw226dvxac7jzcpsf4xb64r4epr6o5hgn46dxlqk7gnjptakik6xnzqd.onion/` * plain: `https://xw226dvxac7jzcpsf4xb64r4epr6o5hgn46dxlqk7gnjptakik6xnzqd.onion/`
* proof: [link](https://mdleom.com) * proof: [link](https://mdleom.com)
* check: <span title="attempts=1 code=200 exit=0 time=2022-04-10 00:15:18+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 20:49:23+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 08:49:26+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-08 17:23:22+00:00">:white_check_mark:</span><span title="attempts=6 code=903 exit=7 time=2022-04-08 05:29:47+00:00">:sos:</span><span title="attempts=1 code=200 exit=0 time=2022-04-07 13:57:27+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-07 01:57:21+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-06 22:32:18+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-06 10:32:26+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 19:06:30+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 07:06:30+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 15:40:31+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 03:40:24+00:00">:white_check_mark:</span><span title="attempts=5 code=200 exit=0 time=2022-04-03 12:28:05+00:00">:white_check_mark:</span> * check: <span title="attempts=1 code=200 exit=0 time=2022-04-10 00:15:18+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 20:49:23+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 08:49:26+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-08 17:23:22+00:00">:white_check_mark:</span><span title="attempts=6 code=903 exit=7 time=2022-04-08 05:29:47+00:00">:question:</span><span title="attempts=1 code=200 exit=0 time=2022-04-07 13:57:27+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-07 01:57:21+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-06 22:32:18+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-06 10:32:26+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 19:06:30+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 07:06:30+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 15:40:31+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 03:40:24+00:00">:white_check_mark:</span><span title="attempts=5 code=200 exit=0 time=2022-04-03 12:28:05+00:00">:white_check_mark:</span>
### [Nick Frichette](https://nickf43ab43xxf3yqgzy5uedsjij6h473rmbyzq6inohcnr3lohlu3yd.onion/) ### [Nick Frichette](https://nickf43ab43xxf3yqgzy5uedsjij6h473rmbyzq6inohcnr3lohlu3yd.onion/)
* transport: :closed_lock_with_key: **HTTPS** * transport: :closed_lock_with_key: **HTTPS**
@ -1115,7 +1115,7 @@ You can find techical details and the legend/key for symbols in the [footnotes s
* link: [http://gppg43zz5d2yfuom3yfmxnnokn3zj4mekt55onlng3zs653ty4fio6qd.onion](http://gppg43zz5d2yfuom3yfmxnnokn3zj4mekt55onlng3zs653ty4fio6qd.onion) * link: [http://gppg43zz5d2yfuom3yfmxnnokn3zj4mekt55onlng3zs653ty4fio6qd.onion](http://gppg43zz5d2yfuom3yfmxnnokn3zj4mekt55onlng3zs653ty4fio6qd.onion)
* plain: `http://gppg43zz5d2yfuom3yfmxnnokn3zj4mekt55onlng3zs653ty4fio6qd.onion` * plain: `http://gppg43zz5d2yfuom3yfmxnnokn3zj4mekt55onlng3zs653ty4fio6qd.onion`
* proof: [link](https://www.cbc.ca/securedrop) * proof: [link](https://www.cbc.ca/securedrop)
* check: <span title="attempts=1 code=200 exit=0 time=2022-04-10 00:17:05+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 20:50:33+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 09:15:37+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-08 17:25:04+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-08 05:25:38+00:00">:white_check_mark:</span><span title="attempts=6 code=903 exit=7 time=2022-04-07 14:04:06+00:00">:sos:</span><span title="attempts=6 code=903 exit=7 time=2022-04-07 02:04:11+00:00">:sos:</span><span title="attempts=6 code=903 exit=7 time=2022-04-06 22:40:19+00:00">:sos:</span><span title="attempts=1 code=200 exit=0 time=2022-04-06 10:34:21+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 19:07:37+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 07:08:10+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 15:41:41+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 03:41:42+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-03 12:18:02+00:00">:white_check_mark:</span> * check: <span title="attempts=1 code=200 exit=0 time=2022-04-10 00:17:05+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 20:50:33+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-09 09:15:37+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-08 17:25:04+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-08 05:25:38+00:00">:white_check_mark:</span><span title="attempts=6 code=903 exit=7 time=2022-04-07 14:04:06+00:00">:question:</span><span title="attempts=6 code=903 exit=7 time=2022-04-07 02:04:11+00:00">:question:</span><span title="attempts=6 code=903 exit=7 time=2022-04-06 22:40:19+00:00">:question:</span><span title="attempts=1 code=200 exit=0 time=2022-04-06 10:34:21+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 19:07:37+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-05 07:08:10+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 15:41:41+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-04 03:41:42+00:00">:white_check_mark:</span><span title="attempts=1 code=200 exit=0 time=2022-04-03 12:18:02+00:00">:white_check_mark:</span>
### [CNN](http://qmifwf762qftydprw2adbg7hs2mkunac5xrz3cb5busaflji3rja5lid.onion) ### [CNN](http://qmifwf762qftydprw2adbg7hs2mkunac5xrz3cb5busaflji3rja5lid.onion)
*via: https://securedrop.org/api/v1/directory/* *via: https://securedrop.org/api/v1/directory/*
@ -1416,6 +1416,7 @@ These sites have apparently stopped responding.
- :sos: site returned no data, or is down, or curl experienced a transient or permanent network error; may also reflect a problem with the RWOS server connection - :sos: site returned no data, or is down, or curl experienced a transient or permanent network error; may also reflect a problem with the RWOS server connection
- :boom: same as :sos: but curl specifically mentioned SSL certificates as an issue - :boom: same as :sos: but curl specifically mentioned SSL certificates as an issue
- :alarm_clock: same as :sos: but curl specifically mentioned timeout as an issue - :alarm_clock: same as :sos: but curl specifically mentioned timeout as an issue
- :question: same as :sos: but curl specifically mentioned inability to resolve an onion descriptor
- :new: site is newly added, no data yet - :new: site is newly added, no data yet
You can also see the [history of updates](https://github.com/alecmuffett/real-world-onion-sites/commits/master/README.md). You can also see the [history of updates](https://github.com/alecmuffett/real-world-onion-sites/commits/master/README.md).

9
rwos-db.py
View File

@ -3,6 +3,7 @@ from datetime import datetime, timezone
from multiprocessing import Pool, Lock from multiprocessing import Pool, Lock
import csv import csv
import datetime as dt import datetime as dt
import re
import sqlite3 import sqlite3
import subprocess import subprocess
import sys import sys
@ -40,6 +41,7 @@ EMOJI_5xx = ':stop_sign:'
EMOJI_DEAD = ':sos:' EMOJI_DEAD = ':sos:'
EMOJI_NO_DATA = ':new:' EMOJI_NO_DATA = ':new:'
EMOJI_BAD_CERT = ':boom:' EMOJI_BAD_CERT = ':boom:'
EMOJI_NO_DESC = ':question:'
EMOJI_TIMED_OUT = ':alarm_clock:' EMOJI_TIMED_OUT = ':alarm_clock:'
H1 = '#' H1 = '#'
@ -251,12 +253,15 @@ def get_summary(url):
elif hcode >= 500 and hcode < 600: elif hcode >= 500 and hcode < 600:
emoji = EMOJI_5xx emoji = EMOJI_5xx
elif hcode >= BADNESS: elif hcode >= BADNESS:
emoji = EMOJI_DEAD # default
if 'SSL certificate' in errstr: if 'SSL certificate' in errstr:
emoji = EMOJI_BAD_CERT emoji = EMOJI_BAD_CERT
elif 'timed out' in errstr: elif 'timed out' in errstr:
emoji = EMOJI_TIMED_OUT emoji = EMOJI_TIMED_OUT
else: elif "Can't complete SOCKS5 connection" in errstr:
emoji = EMOJI_DEAD # todo: parse out socks error codes from https://datatracker.ietf.org/doc/html/rfc1928#section-6
if re.search(r'\(4\)$', errstr):
emoji = EMOJI_NO_DESC
t = datetime.fromtimestamp(when, timezone.utc) t = datetime.fromtimestamp(when, timezone.utc)
result.append('<span title="attempts={1} code={2} exit={3} time={4}">{0}</span>'.format(emoji, attempt, hcode, ecode, t)) result.append('<span title="attempts={1} code={2} exit={3} time={4}">{0}</span>'.format(emoji, attempt, hcode, ecode, t))
return result return result