proofreading + notes
I think the article is in a really good spot, other than any unfinished content. It reads very well, and has a good logical flow to it
This commit is contained in:
parent
7400ab92da
commit
a904e2c772
|
|
@ -21,6 +21,7 @@ Combining a *date* and a *time* gives you an precise instant in time, and is com
|
|||
Learning resources:
|
||||
|
||||
- TODO: uh is there really anything good to link here
|
||||
[NOTE(hayden): I'm not really sure that anything is needed for this section. This is basically just covering the tip of the iceberg that most are already familiar with]
|
||||
|
||||
## Time zones
|
||||
|
||||
|
|
@ -39,8 +40,9 @@ It is worth noting that the time offset may not always be in increments of one h
|
|||
|
||||
Resources:
|
||||
|
||||
- TODO: Definition of UTC
|
||||
- TODO: Definition of UTC [NOTE(hayden): This resource seems alright (same web domain as below, though): https://www.timeanddate.com/time/aboututc.html]
|
||||
- https://www.timeanddate.com/time/time-zones-interesting.html (not a super great resource? find a better one?)
|
||||
* [NOTE(hayden): This seems fairly comprehensive in terms of a reference: https://help.skytap.com/kb-time-zones-and-utc-offsets.html]
|
||||
|
||||
### Aside: UTC and the International Date Line
|
||||
|
||||
|
|
@ -48,20 +50,20 @@ Broadly speaking, UTC (Coordinated Universal Time) is the date and time at the p
|
|||
|
||||
You may have also heard of GMT (Greenwich Mean Time). GMT is also centered at the prime meridian, but is UK-centric and therefore subject to some historical nonsense. UTC is the internationally standard successor, and the one appropriate for computer use.
|
||||
|
||||
The existence of the prime meridian necessitates a date cutoff point at roughly 180 degrees longitude. This is the International Date Line, the point at which you cross from UTC-12 to UTC+12 or vice versa, changing the date as necessary.
|
||||
The existence of the prime meridian necessitates a date cutoff point at roughly 180 degrees longitude. This is the International Date Line, the point at which you cross from UTC-12 to UTC+12 or vice versa, changing the date as necessary. [NOTE(hayden): Not sure how important this is, but an image might be useful here]
|
||||
|
||||
You might think that the International Date Line ensures that no two locations on earth have a time difference greater than 24 hours. But naturally, there are in fact time zones whose time offset is greater than 12 hours from UTC. Rules are made to be broken.
|
||||
|
||||
Resources:
|
||||
|
||||
- TODO: Definition of GMT and how it relates to BST and other British politics
|
||||
- TODO: Definition of the IDL
|
||||
- TODO: Definition of GMT and how it relates to BST and other British politics [NOTE(hayden): The Wikipedia page is fairly brief and covers GMT historically and some legislation related to it: https://en.wikipedia.org/wiki/Greenwich_Mean_Time#GMT_in_legislation]
|
||||
- TODO: Definition of the IDL [NOTE(hayden): Interesting image here: https://oceanservice.noaa.gov/facts/international-date-line.html There's also an image on Wikipedia that may be worth linking to: https://en.wikipedia.org/wiki/International_Date_Line]
|
||||
|
||||
## Daylight saving time and locations
|
||||
|
||||
Many locations on Earth observe regular changes to their timekeeping, shifting their clocks forward or backward by one hour to put sunrise and sunset in a different place relative to their working day. Many people, especially programmers, think this is stupid, but we programmers are stuck with it until every government on Earth decides to abolish this practice.
|
||||
|
||||
The most important implication of daylight saving time is that **you must not assume that a day is 24 hours long.** A calendar day may be 23 or 25 hours long, depending on the direction of the transition. The visible impact of this is that the day will either repeat or skip an hour, and this change may occur at different times of day depending on the location.
|
||||
The most important implication of daylight saving time is that **you must not assume that a day is 24 hours long.** A calendar day may be 23 or 25 hours long, depending on the direction of the transition [NOTE(hayden): Are there any instances in any countries where it is some arbitrary time BETWEEN 23-25 hours, or are the options specifically 23, 24, and 25?]. The visible impact of this is that the day will either repeat or skip an hour, and this change may occur at different times of day depending on the location.
|
||||
|
||||
For example, time in Chicago on March 14, 2021 proceeded directly from 1:59:59 CST (Central Standard Time) to 3:00:00 CDT (Central Daylight Time). Likewise, on November 7, 2021, time proceeded directly from 1:59:59 CDT to 1:00:00 CST. Note that when repeating an hour, the time zone resolves the ambiguity.
|
||||
|
||||
|
|
@ -72,13 +74,14 @@ To help you deal with daylight saving time, you may have heard mnemonics such as
|
|||
| Standard | DST | Spring | "Spring forward" | "Lose an hour" | Traveling east |
|
||||
| DST | Standard | Fall | "Fall back" | "Gain an hour" | Traveling west |
|
||||
|
||||
A useful gut-check for all this is to remember that the Sun rises in the east and sets in the west. Traveling west would thus mean traveling with the Sun and experiencing more daylight—hence, "gaining an hour", and having to adjust your clock _backward_ to account for the extra time in your day.
|
||||
A useful gut-check for all this is to remember that the Sun rises in the east and sets in the west. [NOTE(hayden): Is this just a heuristic? Technically it's a generalization and isn't always the case, but I don't know if that's being pedantic or if it's a useful clarification.] Traveling west would thus mean traveling with the Sun and experiencing more daylight—hence, "gaining an hour", and having to adjust your clock _backward_ to account for the extra time in your day.
|
||||
|
||||
> Aside: there are many conflicting explanations for who originally proposed the practice of daylight saving time, why governments actually chose to adopt this practice, and why it has continued to this day. This article does not attempt to clarify any of this, but the author chooses to believe that all this is Benjamin Franklin's greatest prank of all time.
|
||||
|
||||
Resources:
|
||||
|
||||
- TODO
|
||||
- [NOTE(hayden): Wikipedia has a comprehensive list of countries currently observing and those that previous observed Daylight Saving Time. It also includes the start/end for each country: https://en.wikipedia.org/wiki/Daylight_saving_time_by_country]
|
||||
|
||||
## Leap days and leap seconds
|
||||
|
||||
|
|
@ -99,6 +102,7 @@ The irregular and unpredictable nature of leap seconds has resulted in varying i
|
|||
- https://docs.ntpsec.org/latest/leapsmear.html
|
||||
|
||||
[NOTE(ben): It kind of feels like we should have covered NTP in some limited way by now?]
|
||||
[NOTE(hayden): I think the logical flow for when it occurs below makes sense]
|
||||
|
||||
Resources:
|
||||
|
||||
|
|
@ -107,7 +111,7 @@ Resources:
|
|||
|
||||
## Date representations, the Unix epoch, and the year 2038 problem
|
||||
|
||||
With all these concepts established, the question now is how to actually represent these in your application. For most uses of eeeboo beeboo
|
||||
With all these concepts established, the question now is how to actually represent these in your application. For most uses of eeeboo beeboo [NOTE(hayden): Wat xD]
|
||||
|
||||
- Unix timestamp
|
||||
- Seconds since January 1, 1970 UTC
|
||||
|
|
@ -119,8 +123,8 @@ With all these concepts established, the question now is how to actually represe
|
|||
- Fractional seconds since January 1, 1900 UTC (even if UTC didn't exist back then, see above).
|
||||
- Uses 64 bit fixed point format. The 32 most significant bits denote the number of seconds, the 32 least significant bits denote fractional seconds. The smallest time increment is thus $2^{-32}$ seconds (232 picoseconds). The maximum representable time range is approximately 136 years.
|
||||
- NTP date
|
||||
- It uses 128 bits as follows: a 32 bits signed Era number, a 96 fixed point era offset (32 bits for seconds and 64 bits for fractional seconds).
|
||||
- Covers the whole universe existence with a precision well below any duration that can be directly measured.
|
||||
- It uses 128 bits as follows: a 32 bits signed Era number, and a 96 fixed point era offset (32 bits for seconds and 64 bits for fractional seconds).
|
||||
- Covers the whole universe's existence with a precision well below any duration that can be directly measured.
|
||||
- RFC 3339 date-time
|
||||
- Unambiguously stores an instant in time
|
||||
- Indicates offset from UTC (but _not_ time zone in the proper sense - no DST)
|
||||
|
|
@ -128,7 +132,6 @@ With all these concepts established, the question now is how to actually represe
|
|||
- Restricted subset of ISO 8601
|
||||
- You literally have no reason to use ISO 8601. Just don't. A copy of the spec costs hundreds of dollars, and it allows way more formats than anyone needs. Anyone who claims to accept "ISO 8601 timestamps" is wrong; they probably actually support RFC 3339 and a few variants. Meanwhile, RFC 3339 is freely available and has a small but complete set of features.
|
||||
- Contains useful definitions for both date and time encoding
|
||||
-
|
||||
|
||||
| | | |
|
||||
| -------------- | ---- | ---- |
|
||||
|
|
@ -153,7 +156,7 @@ Getting the time seems easy right? Someone has it, you ask them, and you're done
|
|||
|
||||
Let's say you want to set your watch, so you ask the time to a friend. But also imagine voice takes a long, undeterminate amount of time to travel between you and them (alternatively, imagine you want to know what time is it with an extreme precision). When your friend hears your question, it is already too late. They can give you the time it was when the heard the question, but they have no idea when you asked. Now, when you receive their answer you have the same problem. You neither know when they heard your question, nor when they answered. You only know that the time you get corresponds to some point between the moment you asked and the moment you heard the answer.
|
||||
|
||||
Now let's say you pick a random point in this interval, and adjust your own watch to that time. Maybe you make an error, but at least your synchronized with your friend _within some tolerance_ right? No luck! Since it is extremely unlikely both of your watches have exactly the same consistent speed, your watch will drift and wander away from your friend's clock. You could ask again and again. But each time you will get a different error, so your watch will be very noisy (and still drift in between exchanges). Furthermore it seems like a big waste of your... time.
|
||||
Now let's say you pick a random point in this interval, and adjust your own watch to that time. Maybe you make an error, but at least you're synchronized with your friend _within some tolerance_ right? No luck! Since it is extremely unlikely both of your watches have exactly the same consistent speed, your watch will drift and wander away from your friend's clock. You could ask again and again. But each time you will get a different error, so your watch will be very noisy (and still drift in between exchanges). Furthermore it seems like a big waste of your... time.
|
||||
|
||||
Fortunately for us humans, we don't typically require a high precision compared to the time it takes for sound to travel between two persons speaking to each other. For computers though, it can be crucial to be synchronized within a few microseconds, and network communications can take hundreds of milliseconds. Their clock are also drifting and jittery, sometimes at an astonishingly high rate. Clearly there has to be ways to mitigate the errors generated by such adverse conditions.
|
||||
|
||||
|
|
@ -167,11 +170,12 @@ Other clock synchronization protocols exist, e.g. PTP (IEEE 1588-2019), and some
|
|||
I will focus here on NTP since it's the most documented and widely used, and it doesn't require specialized hardware like PTP.
|
||||
|
||||
## The NTP Network
|
||||
|
||||
To maintain a consistent notion of time accross the internet, you first need some reference time signal, which is typically provided by atomic clocks. Since it is neiter practical nor desirable to connect every computer in the world to these atomic clocks, the time signal is distributed accross a network of peers organized in serveral strata. Primary servers sit at stratum 1 and are directly connected (through wire or radio) to a reference clock. Peers at strata 2 can synchronize on servers at strata 1 or 2, and act as a synchronization sources for peers at strata 3. The layering continues up to stratum 15. Strata 16 represents unsynchronized devices.
|
||||
|
||||
|
||||
|
||||
The protocol selects sources in order to avoid loops and minimize the round-trip delay to a primary server. As such peers collaborate to automatically reorganize the network to produce the most accurate time, even in the presence of failures in the network.
|
||||
The protocol selects sources in order to avoid loops and minimize the round-trip delay to a primary server. As such, peers collaborate to automatically reorganize the network to produce the most accurate time, even in the presence of failures in the network.
|
||||
|
||||
## NTP Architecture
|
||||
|
||||
|
|
@ -180,11 +184,12 @@ Now let's see the implementation of one node of the network. The following diagr
|
|||
|
||||
|
||||
### On-Wire Protocol
|
||||
|
||||
The first stage consists of getting time estimates from each peer. In the simplest mode of operations, the node polls a peer by sending a request packet timestamped with the transmission time, $t_1$. Upon reception of the request, the peer stores a timestamp $t_2$, processes the message, and sends a response packet containing $t_1$, $t_2$ and the response transmission time $t_3$. The node receives the response at time $t_4$.
|
||||
|
||||
|
||||
|
||||
The node then compute the tuple $(\delta, \theta, \epsilon)$, where:
|
||||
The node then computes the tuple $(\delta, \theta, \epsilon)$, where:
|
||||
|
||||
- $\delta$ is the round-trip delay.
|
||||
- $\theta$ is the estimated offset of the server clock with respect to the local clock.
|
||||
|
|
@ -198,18 +203,18 @@ $\begin{align*}
|
|||
|
||||
Note that peers also transmit the delay and dispersion statistics accumulated at each stratum from the root down to the peer, yielding the root delay and dispersion, $\theta_r$ and $\varepsilon_r$. These statistics are used to assess the "quality" of the samples computed from each peer.
|
||||
|
||||
Simple NTP clients that are only synchronized to one server, that don't act as a source for other peers, and that don't need a high precision, can implement only the on-wire protocol and directly use $\theta$ to correct their local clock. However this estimate is vulnerable to network jitter, delay asymetry, clock drift and wander. To get better precision, the samples produced by the on-wire protocol must be processed by a number of mitigation algorithms.
|
||||
Simple NTP clients that are only synchronized to one server, that don't act as a source for other peers, and that don't need a high precision, can implement only the on-wire protocol and directly use $\theta$ to correct their local clock. However this estimate is vulnerable to network jitter, delay asymmetry, clock drift and wander. To get better precision, the samples produced by the on-wire protocol must be processed by a number of mitigation algorithms.
|
||||
|
||||
Links: [Mills, Analysis and Simulation of the NTP On-Wire Protocols](https://www.eecis.udel.edu/~mills/onwire.html), [RFC5905 - section 8](https://datatracker.ietf.org/doc/html/rfc5905#section-8), [SNTP (RFC4330)](https://datatracker.ietf.org/doc/html/rfc4330)
|
||||
|
||||
### Clock Filter
|
||||
|
||||
The clock filter is a shift register containing the last 8 samples produced by the on-wire protocol. The filter algorithm select the sample with maximum expected accuracy. It is based on the observations that:
|
||||
The clock filter is a shift register containing the last 8 samples produced by the on-wire protocol. The filter algorithm selects the sample with maximum expected accuracy. It is based on the observations that:
|
||||
|
||||
* The samples with the low network delay were likely to exhibit a low error at the time they were produced.
|
||||
* The samples with the low [NOTE(hayden): is this meant to read 'lowest'?] network delay were likely to exhibit a low error at the time they were produced.
|
||||
* Older samples carry more accuracy than newer ones due to clock drift.
|
||||
|
||||
As a new sample is pushed to the register, it evicts the oldest sample. The dispersion values of the other samples are the incremented by $\varphi_c\times\Delta_t$, where $\Delta_t$ is the time elapsed since the last update from the server. A distance metric $\lambda_i$ is affected to each sample:
|
||||
As a new sample is pushed to the register, it evicts the oldest sample. The dispersion values of the other samples are then incremented by $\varphi_c\times\Delta_t$, where $\Delta_t$ is the time elapsed since the last update from the server. A distance metric $\lambda_i$ is affected [NOTE(hayden): as in appended?] to each sample:
|
||||
|
||||
$\lambda_i = \frac{\delta_i}{2}+\varepsilon_i \,.$
|
||||
|
||||
|
|
@ -219,7 +224,7 @@ The peer's dispersion $\varepsilon_p$ is computed as an exponentially weighted a
|
|||
|
||||
$\varepsilon_p = \sum_{i=0}^{N-1}\frac{\varepsilon_i}{2^{(i+1)}} \,.$
|
||||
|
||||
The server's jitter $\psi_p$ is computed as the root mean square (RMS) of the offsets differences with respect to the offset of the first sample's in the list:
|
||||
The server's jitter $\psi_p$ is computed as the root mean square (RMS) of the offsets differences with respect to the offset of the first sample's in the list [NOTE(hayden): This sentence is difficult to read -- it may be because of the multiple forms of gramatical possessiveness. There also might be some typos in that regard]:
|
||||
|
||||
$\psi_p = \frac{1}{n-1}\sqrt{\sum_{i=0}^{N-1}(\theta_0-\theta_i)^2} \,.$
|
||||
|
||||
|
|
@ -239,11 +244,11 @@ Links: [Mills, Clock Select Algorithm](https://www.eecis.udel.edu/~mills/ntp/htm
|
|||
|
||||
Once falsetickers have been eliminated, truechimers are placed in a survivor list, that is pruned by the cluster algorithm in a series of rounds.
|
||||
|
||||
For each candidate, it compute the selection jitter, which is the root mean square of offset differences between this candidate and all other peers. Candidates are ranked by their selection jitter and the candidate with the greatest one is pruned from the survivor list.
|
||||
For each candidate, it computes the selection jitter, which is the root mean square of offset differences between this candidate and all other peers. Candidates are ranked by their selection jitter and the candidate with the greatest one is pruned from the survivor list.
|
||||
|
||||
The cluster algorithm continues until a specified minimum of survivors remain, or until the maximum selection jitter is less than the minimum peer jitter (in which case removing the outlier wouldn't improve the accuracy of the selection).
|
||||
|
||||
To get some grasp of _why_ it is designed that way, imagine all samples are coming from the same "idealized" peer, and that we select one "most likely accurate" sample. Then as we already saw in the clock filter algorithm, we can compute the jitter of that idealized peer. The selection jitter is essentially a measure of the jitter implied by the decision to consider that sample the most accurate. Now to understand the termination condition, remember that each sample is produced by a peer that itself exhibits some jitter. There is no point in trying to get a better jitter from the combination of all samples, than the minimum peer jitter.
|
||||
To get some grasp of _why_ it is designed that way, imagine all samples are coming from the same "idealized" peer, and that we select one "most likely accurate" sample. Then, as we already saw in the clock filter algorithm, we can compute the jitter of that idealized peer. The selection jitter is essentially a measure of the jitter implied by the decision to consider that sample the most accurate. Now to understand the termination condition, remember that each sample is produced by a peer that itself exhibits some jitter. There is no point in trying to get a better jitter from the combination of all samples, than the minimum peer jitter.
|
||||
|
||||
Links: [Mills, Cluster Algorithm](https://www.eecis.udel.edu/~mills/ntp/html/cluster.html), [RFC5905 - section 11.2.2](https://datatracker.ietf.org/doc/html/rfc5905#section-11.2.2)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue