Thursday, March 21, 2024

Why can't you hardcode NTP IP???

UPDATE (21 March 2024): Some relevant text from RFC8633:

https://www.rfc-editor.org/rfc/rfc8633.html#section-7

   Note well that using a single anycast address for NTP presents its
   own potential issues.  It means each client will likely use a single
   time server source.  A key element of a robust NTP deployment is each
   client using multiple sources of time.  With multiple time sources, a
   client will analyze the various time sources, select good ones, and
   disregard poor ones.  If a single anycast address is used, this
   analysis will not happen.  This can be mitigated by creating
   multiple, separate anycast pools so clients can have multiple sources
   of time while still gaining the configuration benefits of the anycast
   pools.

   If clients are connected to an NTP server via anycast, the client
   does not know which particular server they are connected to.  As
   anycast servers enter and leave the network or the network topology
   changes, the server to which a particular client is connected may
   change.  This may cause a small shift in time from the perspective of
   the client when the server to which it is connected changes.  Extreme
   cases where the network topology changes rapidly could cause the
   server seen by a client to rapidly change as well, which can lead to
   larger time inaccuracies.  It is RECOMMENDED that network operators
   only deploy anycast NTP in environments where operators know these
   small shifts can be tolerated by the applications running on the
   clients being synchronized in this manner.

 

UPDATE (21 March 2024): Some hacky workarounds: You can probably hardcode these IPs, though there is absolutely no guarantee that they will continue to work:

miyuru on Dec 30, 2022 | prev | next [–]

> It would be great to see Google or Cloudflare use their infrastructure to provide anycasted NTP IP addresses.

Google, Cloudflare and Facebook has vanity IPv6 address, pretty sure they are all static anycast IPs.

time.google.com - 2001:4860:4806::

time.cloudflare.com - 2606:4700:f1::123

time.facebook.com - 2a03:2880:ff0c::123 


jedisct1 on Dec 30, 2022 | parent | prev | next [–]

As for IPv4, time.google.com has been 216.239.35.0 since 2016, so it's unlikely to change anytime soon either.


I can confirm that time.google.com still resolves to that IP address. I also ran these commands today (21 March 2024) for recordkeeping purposes:

$ host time.facebook.com
time.facebook.com has address 129.134.29.123


$ host time.cloudflare.com
time.cloudflare.com has address 162.159.200.123
time.cloudflare.com has address 162.159.200.1


UPDATE (21 March 2024): Still no viable solutions, see below.

UPDATE: I see that there are already-existing solutions for the problem I described:

  • tlsdate - https://github.com/ioerror/tlsdate       (but see below)
  • roughtime proposal - https://datatracker.ietf.org/doc/html/draft-ietf-ntp-roughtime

UPDATE: Here's a relevant blog post by Hanno Bock: https://blog.hboeck.de/plugin/tag/tlsdate

tlsdate is a hack abusing the timestamp of the TLS protocol. The TLS timestamp of a server can be used to set the system time. This doesn't provide high accuracy, as the timestamp is only given in seconds, but it's good enough.

I've used and advocated tlsdate for a while, but it has some problems. The timestamp in the TLS handshake doesn't really have any meaning within the protocol, so several implementers decided to replace it with a random value. Unfortunately that is also true for the default server hardcoded into tlsdate.

Some Linux distributions still ship a package with a default server that will send random timestamps. The result is that your system time is set to a random value. I reported this to Ubuntu a while ago. It never got fixed, however the latest Ubuntu version Zesty Zapis (17.04) doesn't ship tlsdate any more.

Given that Google has shipped tlsdate for some in ChromeOS time it seems unlikely that Google will send randomized timestamps any time soon. Thus if you use tlsdate with www.google.com it should work for now. But it's no future-proof solution.

TLS 1.3 removes the TLS timestamp, so this whole concept isn't future-proof. Alternatively it supports using an HTTPS timestamp. The development of tlsdate has stalled, it hasn't seen any updates lately. It doesn't build with the latest version of OpenSSL (1.1) So it likely will become unusable soon.

Roughtime

Roughtime is a Google project. It fetches the time from multiple servers and uses some fancy cryptography to make sure that malicious servers get detected. If a roughtime server sends a bad time then the client gets a cryptographic proof of the malicious behavior, making it possible to blame and shame rogue servers. Roughtime doesn't provide the high accuracy that NTP provides.

From a security perspective it's the nicest of all solutions. However it fails the availability test. Google provides two reference implementations in C++ and in Go, but it's not packaged for any major Linux distribution. Google has an unfortunate tendency to use unusual dependencies and arcane build systems nobody else uses, so packaging it comes with some challenges.

But wait, it looks like roughtime also requires DNS? At least I haven't been able to find any roughtime IPs that I can hardcode. 





 

 

Original post:

People online say that you shouldn't hardcode NTP IPs, but I don't see why this has to be the case. 

You can hardcode 1.1.1.1 for DNS, so why can't you hardcode an IP for NTP? 

People online say that the NTP server might go down, but that shouldn't be an issue because IP anycast will automatically route the traffic to the nearest available server.

People online say that you might overload the server, but you can do load balancing internally within your datacenter in any number of ways, so that shouldn't be an issue either.

You can argue that IP anycast won't work because the packets might get redirected to another server, but this happens so rarely in practice that it shouldn't be a problem, and you can just try again if it fails.

I don't see what's so special about NTP that you can't have an anycast IP for it like 1.1.1.1

I am writing this blog post because TLS won't work if your clock is wrong. If you force your machine to only use DNS-over-HTTPS, then you can't resolve any domains if your cloick is wrong. 

So this leads to a catch-22 situation: Your DNS doesn't work because your clock is wrong, and you can't fix your clock because you can't resolve NTP domain names to IP addresses because your DNS doesn't work.

This problem would be solved if we could hardcode an IP address for NTP just like we can do with DNS (1.1.1.1)

EDIT: I see that someone has already made a blog post on this: https://news.ycombinator.com/item?id=34177331


> Alternatively it would be good to use an anycast IP for NTP. This is normally a bad idea because it makes calculating skew hard/unreliable, but that really should just mean a poorly sync'ed clock. So set the Anycast clock to be an intentionally high/poor Stratum score, list this along with a DNS based address so it's used until the encrypted DNS can be resolved with a better Stratum score. -- Bob H

Yes, so I suppose anycast might cause poor skew, though that isn't a problem for this use case because TLS will work even if your clock is a few minutes wrong. 

But I suppose we could create a simpler version of NTP whose purpose is to just set your clock to some good-enough-for-TLS time, and then switch to actual NTP once your DNS works.


No comments:

Post a Comment