BBRv1 provides much better throughput in many cases and is particularly
useful for our update servers. The fairness issues based on round trip
time are not a major issue for us. The fairness issues for competing
with traditional loss-based congestion control are relevant to us but it
seems to benefit it more than it hurts us. BBRv3 will fix most of this
while preserving nearly all the benefits and will likely be shipped as a
replacement for BBRv1 in the Linux kernel rather than another option.
The reason we rolled it back last time was seeing cases of the initial
bandwidth estimate being overly low combined with a very bad interaction
with synproxy causing low bandwidth initially. We've partially addressed
the synproxy issue by raising the synproxy threshold based on conntrack
table size which we're now fully scaling based on available memory. If
we decide this is still a significant issue, we can limit using BBRv1 to
our update servers where it has massive benefits and the least downside
due to initial bandwidth not being as important. BBRv3 will help with
this by probing Round Trip Time every 5 seconds instead of 10 seconds
but still has similar issues.
We can no longer use OCSP stapling and Must-Staple. These will soon be
obsolete once the `shortlived` profile is available for public use since
it will provide certificates with a similar lifetime as OCSP responses.
In the meantime, we've moved to the `tlsserver` profile stripping legacy
features to prepare for the `shortlived` profile which will be identical
to `tlsserver` but with a validity period of 6 days.
The certificate for SUPL is still temporarily using the classic profile
to work around the older generations of end-of-life Snapdragon Pixels
not having support for SNI. We can eventually drop support for these
devices from the SUPL service to allow us to disable TLSv1.1, DHE and
move to the `tlsserver` or `shortlived` profile.
The certificate for SMTP is still temporarily using the classic profile
to avoid potential compatibility issues with servers supporting TLSv1.2
but still not yet supporting SNI.
BBRv1 significantly improves throughput in some cases but it also
significantly reduces it in others. We've run into too many network
conditions it handles quite poorly. There's also a bad interaction
between BBR and synproxy where it will cripple the initial throughput
for connections established via synproxy. This means a basic SYN flood
attack could cripple initial TCP throughput for most connections.
Android doesn't enable ECN for outbound connections yet and we don't
want to deviate from that so it mainly only gets activated for macOS
and iOS clients. Linux kernel approach to ECN hasn't been modernized and
there are fierce debates about how it should work. It can cause issues
and it seems best to avoid it until Android enables it.
