Daniel Micay
9d68a079db
logrotate: use specific log file paths
...
This avoids ending up with the glob path in the logrotate state file
when nothing matches the glob pattern.
2025-11-03 12:54:18 -05:00
Daniel Micay
39b6de58dd
syslog-ng: add socket for nginx error logs
...
The error log is fairly quiet during regular use but can end up logging
one or more lines per request during DDoS attacks. Errors are logged for
worker_connections depletion and limit_conn rejections. There's also
currently an nginx bug with modern TLS and OpenSSL causing some client
side TLS errors to be logged as crit instead of info.
2025-11-03 12:53:24 -05:00
Daniel Micay
386d332aaf
remove unused logrotate configurations
2025-11-03 00:33:30 -05:00
Daniel Micay
ca20c421a5
deploy-certbot: avoid syncing replicate.conf
2025-11-03 00:33:30 -05:00
Daniel Micay
934c5dbd53
logrotate: remove notifempty for nginx
2025-11-03 00:33:30 -05:00
Daniel Micay
b61c76c324
logrotate: remove nocreate for letsencrypt
2025-11-03 00:33:30 -05:00
Daniel Micay
cee00863e3
update servers haven't been on OVH for a while
2025-11-03 00:33:30 -05:00
Daniel Micay
39e701e9fb
update pacreport.conf
2025-11-03 00:33:30 -05:00
Daniel Micay
944b4679c1
merge website and network servers
...
This provides more redundancy for both services through having 2
instances in each region. The network services have much higher
bandwidth usage and load so this will also delay us needing to obtain
new servers by making better use of the ones we have.
2025-11-03 00:33:30 -05:00
Daniel Micay
2caa67529a
set up syslog-ng for nginx access log
...
This sets up the infrastructure for moving from storing nginx access
logs in journald to plain text files written by syslog-ng and rotated by
logrotate. This works around the poor performance, poor space efficiency
and lack of archived log compression for journald. Unlike writing access
logs directly with nginx, this continues avoiding blocking writes in the
event loop and sticks to asynchronous sends through a socket.
Since nginx only supports syslog via the RFC 3164 protocol rather than
the more modern RFC 5424 protocol, this leaves formatting timestamps up
to nginx rather than using the ones provided via the syslog protocol.
2025-11-03 00:33:28 -05:00
Daniel Micay
97d55a130e
enable web services immediately
2025-11-01 20:14:35 -04:00
Daniel Micay
48bdeb4033
explicitly refer to nginx.service
2025-11-01 20:05:50 -04:00
Daniel Micay
3c4380370e
logrotate: use zstd for compression
2025-11-01 20:04:53 -04:00
Daniel Micay
a346146625
reorder update servers
2025-11-01 20:04:51 -04:00
Daniel Micay
01305667bd
remove legacy 2.releases.grapheneos.org IPv6 address
2025-10-31 00:38:22 -04:00
Daniel Micay
7fa179260f
phase in new IPv6 address for 2.releases.grapheneos.org
2025-10-30 20:11:17 -04:00
Daniel Micay
4e771284f5
expand pacreport.conf
2025-10-30 17:09:11 -04:00
Daniel Micay
34a18b6a86
simplify deploy-primary
2025-10-30 17:06:07 -04:00
Daniel Micay
0d1705320f
use consistent naming for session ticket key scripts/units
2025-10-30 17:06:07 -04:00
Daniel Micay
768cc9ada3
update LS_COLORS configuration
...
This is generated from the current standard dircolors database with the
addition of Brotli to the archive file types.
2025-10-30 16:17:33 -04:00
Daniel Micay
9fde84c877
add initial session ticket key synchronization
2025-10-30 14:22:55 -04:00
Daniel Micay
f9430a1aeb
add script for deploying certbot replication setup
2025-10-30 14:22:32 -04:00
Daniel Micay
e6db6a15e6
add swap device timeout as a fallback
...
The previous commit works around a long term systemd bug which recently
began impacting us again. If the workaround stops working, the behavior
should not be stalling boot forever. Swap isn't needed for our servers
to function so it shouldn't break them if it can't be set up.
2025-10-29 22:47:01 -04:00
Daniel Micay
8340cf2813
add workaround for system encrypted swap race
...
This appeared to be solved a while ago but ended up returning.
2025-10-29 22:36:11 -04:00
Daniel Micay
2d87e13eb0
add bootloader deployment script
2025-10-29 18:32:52 -04:00
Daniel Micay
85c5ccc613
update IP addresses for 0.releases.grapheneos.org
2025-10-28 15:25:16 -04:00
Daniel Micay
848d4822e1
rotate-session-ticket-keys: replace is-enabled with is-active
2025-10-28 12:49:17 -04:00
Daniel Micay
0b519d6f5e
set AccuracySec=1us for tcp-fastopen-rotate-keys
2025-10-28 12:33:10 -04:00
Daniel Micay
81147f1fbb
rotate-session-ticket-keys: skip when synced
2025-10-28 02:59:00 -04:00
Daniel Micay
17f0ec527d
cleanly phase in new TLS session ticket keys
...
This closes a small window where new workers could give keys not
accepted by the old workers before they're gracefully shut down. This
will also be needed when syncing keys across a cluster.
2025-10-28 02:48:49 -04:00
Daniel Micay
9ffcb3e648
minor rotate-session-ticket-keys improvements
2025-10-27 23:16:36 -04:00
Daniel Micay
9ed61cef61
reduce TLS session ticket key interval from 8h to 6h
2025-10-27 22:50:32 -04:00
Daniel Micay
ce0942702e
add RemainAfterExit=yes to create-session-ticket-keys.service
2025-10-27 22:11:22 -04:00
Daniel Micay
448565de54
update description for rotate-session-ticket-keys.timer
2025-10-27 21:19:32 -04:00
Daniel Micay
c4af821eda
always create /var/cache/nginx for web servers
...
This avoids needing to restart nginx for ReadWritePaths to kick in after
creating it.
2025-10-27 20:52:34 -04:00
Daniel Micay
fb9e4d6769
remove imagemagick package from mastodon.social
...
This was replaced by libvips.
2025-10-24 15:04:39 -04:00
Daniel Micay
048ccb3fba
allow powerdns user to query pdns over loopback
...
This is being used by the pdns-trigger-health-checks script.
2025-10-23 14:11:56 -04:00
Daniel Micay
9c2183c794
stop blacklisting tls module
...
It no longer gets autoloaded by default due to Linux kernel changes.
2025-10-22 17:36:06 -04:00
Daniel Micay
178791ffd8
update pacreport.conf
2025-10-21 14:11:46 -04:00
Daniel Micay
d6823c9ae0
update python dependencies
2025-10-21 13:21:17 -04:00
Daniel Micay
92288293d9
gitignore: add /lock
2025-10-20 21:46:56 -04:00
Daniel Micay
f8a1d381e7
mdmonitor.service: use syslog reporting
2025-10-19 16:16:33 -04:00
Daniel Micay
e626d67dc1
add nftables deployment script
2025-10-19 15:35:13 -04:00
Daniel Micay
11c9421c63
rename deploy-initial to deploy-initial-vps
2025-10-19 14:58:36 -04:00
Daniel Micay
aaf63a7d87
add 3.releases.grapheneos.org package list
2025-10-19 12:35:01 -04:00
Daniel Micay
04d0489e57
add certbot and web deployment scripts
2025-10-19 12:30:53 -04:00
Daniel Micay
e84c84db6b
disconnect: handle separate non-interactive sessions too
2025-10-11 18:04:48 -04:00
Daniel Micay
f2a4df1d0f
add another IPv6 address for 0.releases.grapheneos.org
...
This will be used to send more traffic to it via DNS RRset load
balancing.
2025-10-11 15:31:09 -04:00
Daniel Micay
5ea8e202a1
0.releases.grapheneos.org IPv4 update
...
The main IPv4 address has changed and we're now using an additional IPv4
address to send more traffic to it via DNS RRset load balancing.
2025-10-11 15:30:35 -04:00
Daniel Micay
02b7e4e5c1
add 3.releases.grapheneos.org server
2025-10-09 09:06:31 -04:00
