Introduction to PTP. How does NTP time synchronization protocol differ from SNTP? The ntp protocol provides precision synchronization mechanisms
07/09/2012, Mon, 10:07, Moscow time
The main problem with next-generation transport networks is that Ethernet technology was originally designed for local area networks and was never intended to carry synchronization signals. In recent decades, circuit-switched networks have been dominated by Synchronous Digital Hierarchy (SDH) technology as a transport medium, based on the transmission of clock signals. But even this reliable and proven technology does not meet the requirements of modern applications.
pages: previous | | 2
Using the Sync Ethernet standard
Ethernet technology was originally developed exclusively for use in local area networks. Methods for linear coding of information at the physical level were selected in accordance with tasks that did not involve transmitting a clock signal. SDH networks initially used NRZ line codes, which are adapted to transmit synchronization at the physical layer of the communication channel. When creating Sync Ethernet technology, the physical layer and coding methods were borrowed from SDH technology, and the second (channel) layer was practically not affected. The frame structure remains unchanged, with the exception of the SSM synchronization status byte. Its meanings were also borrowed from SDH technology.
The principle of synchronization transmission via the Sync Ethernet protocol
The advantages of Sync Ethernet technology include the use of the SDH physical layer structure, and along with this, vast and invaluable experience in designing and building clock network synchronization networks. The identity of the methods has kept the old recommendations G.803, G.804, G.811, G.812 and G.813 relevant in the new technology. Expensive devices - primary reference oscillators (PEGs), secondary master oscillators (MSOs) - can also be used in the new transport network built on the Sync Ethernet standard.
Typical synchronization scheme using Sync Ethernet technology
The disadvantages include the fact that in the entire transmission network, every device must support the new standard, and if there is a device on the line that does not support Sync Ethernet, then all the devices behind this node cannot operate in synchronous mode. Consequently, large material costs are required to modernize the entire network. Another disadvantage is that this method only supports the transmission of frequency synchronization.
Using PTP (IEEE1588v2)
And the last method of transferring synchronization, which has recently become increasingly popular, is the Precise Time Protocol (PTP). It is described in IEEE Recommendation 1588. In 2008, a second version of this document was released, which describes the use of the protocol in telecommunications networks. Precise Time Protocol is quite young, but the time transfer technology itself was borrowed from the Network Time Portocol (NTP) protocol. The NTP protocol in its latest version does not provide the precision required for modern applications, and therefore it remains a good tool for time synchronization, which is widely used in synchronizing servers, distributed databases, etc. But in building a clock network synchronization network, a logical continuation of the NTP protocol is suitable - this is the PTP protocol. The network elements that participate in interaction via the PTP protocol are the following devices: PTP Grand Master and PTP Slave. Typically, the Grand Master takes the timing from the GNSS receiver and, using this information, exchanges packets with the Slave device and constantly corrects time discrepancies between the Grand Master and Slave devices. The more active this exchange is, the higher the accuracy of the adjustment will be. The downside of such active exchange is the increase in bandwidth that is allocated to the PTP protocol. The most important problem in calculating the discrepancy in time intervals is that between the Grand Master and Slave devices there may be “classic” layer 3 routers. The term “classic” in this case is used to emphasize that these devices do not understand anything about the PTP layer 5 protocol.
Delays in the buffers of such routers are quite difficult to manage, and they are random in nature. In order to control these random errors, and also to make the calculation of the time discrepancy between the Grand Master and Slave more accurate, a special parameter was introduced in the PTP protocol - the Time Stamp. This label indicates the time it takes for a packet to pass through the router. If along the entire path from Grand Master to Slave routers have PTP functionality and set a timestamp, then the random error associated with the passage of PTP packets through the IP network can be minimized.
An example of building a synchronization network using the PTP protocol
Comparison of synchronization transfer methods in new generation packet networks
PTP functionality on routers is not required, but is highly recommended when using the PTP protocol. It should be noted that most router manufacturers include this functionality in their devices. An example of constructing a synchronization scheme for a mobile operator is shown in the figure below. The advantage of PTP is that the protocol is designed to transmit all three types of synchronization: frequency, phase and time. The main disadvantage of the protocol is its dependence on the load. When there are congestions on an IP network that are difficult to manage, it is very difficult to ensure strict compliance with the rules for transmitting synchronization across the network.
| Technology | Advantages | Flaws |
| GNSS | Providing frequency, phase and time synchronization. Does not depend on network load. |
Mandatory antenna installation. Impossibility of use in enclosed spaces. Possible interference from other radio devices. Redundancy is only provided by installing a second GNSS receiver |
| Sync Ethernet | Does not depend on network load. Similar to SDH network | Provides only frequency synchronization. Sync Ethernet support is required for all network elements |
| PTP | Providing frequency, phase and time synchronization. | Depends on network load. |
Each method has its own advantages and disadvantages, which are shown in the table. To determine the right approach, it is recommended to consider many criteria that are specific to different networks.
Mikhail Vekselman
pages: previous | | 2
T. Schossig; B. Baumgartner, C. Riesch, M. Rudigier, OMICRON electronics, for website
ANNOTATION
This article discusses general issues related to the Precise Time Protocol described in the IEEE 1588-2008 standard, and also provides the latest information on synchronization and time signal transmission issues that are currently relevant in the power industry. The article also provides an overview of the main issues of the IEEE C37.238-2011 standard, the task of which is to integrate synchronization using the Precise Time Protocol in modern power plants. One of the sections is devoted to the problems of implementation and transition to a new synchronization standard at power facilities, including the network infrastructure requirements that must be met for the successful use of synchronization using the Precise Time Protocol. At the end, a summary of all the issues is given and the prospects for implementing time synchronization in the electric power industry are considered.
INTRODUCTION
The implementation of the IEC 61850 standard is currently being carried out at many electrical power facilities; In this regard, the network infrastructure at substations is undergoing significant modernization. In most cases, communication between multifunction devices (MFPs) in a substation or between an MFP and the main controller is carried out over Ethernet networks. Thus, it is logical to argue that synchronization of all these devices should be carried out over the same network infrastructure, thereby avoiding the creation of additional channels for time synchronization signals. The first step in this direction was the creation of the Network Time Protocol (NTP) within the framework of the IEC 61850 standard, which uses the Ethernet network to transmit time signals. It is known that the NTP protocol allows for synchronization with millisecond accuracy, but often in practice more accurate synchronization is required, so more accurate methods (for example, IRIG-B) must be used in parallel. As a result, there is a need for additional synchronization channels.
The Precise Time Protocol is the first method that allows the use of a synchronization signal over an Ethernet network in substations with the required accuracy in a safe mode. This protocol provides accuracy above 1 μs and can be used for any automation devices of power facilities.
TIME SYNCHRONIZATION IN MODERN SUBSTATIONS
Before we begin to consider the main functions and benefits of the IEEE 1588-2008 protocol, it is necessary to determine the technical requirements for methods of synchronization and time transfer in modern power facilities. This section also provides a brief overview of the main synchronization methods currently in use.
Time measurement accuracy requirements
Due to the fact that all processes and events at power facilities (eg, substations) are controlled from one main center, the absolute accuracy of the system time of the power facility is not so important. However, the synchronous time of each object (substation) becomes of great importance, since the system switches off and on power equipment at different objects, which must be synchronized.
The cascading power outage in the northern United States in August 2003 is a prime example of how difficult post-disaster analysis can be when data on the timing of events is inaccurate. As a result, after this accident, the expert committee investigating the situation demanded the introduction of guidelines defining the minimum absolute accuracy of oscillographed emergency events at facilities. With the adoption of the NERC (North American Electrical Reliability Cooperation) PRC-018-1 standard in 2006, the United States has become required to ensure that all oscilloscope data have a time accuracy of 2 ms or greater relative to UTC (Universal Coordinated Time Scale).
Today, many measurements and control data in power systems must provide absolute timing accuracy of the order of 1 ms:
- data from SCADA systems (Supervisory Control & Data Acqusition)
- data from event recorders and oscilloscopes
- time stamps from MFPs and protection terminals
- lightning measurements
Accuracy of about 1 ms is relatively easy to achieve. However, a range of data in modern devices requires higher accuracy. The following functions require absolute accuracy of 1 µs or better:
- Sample values
- Synchrophasor measurements (Synchronized sine wave vector measurements)
- OMF measurements using the wave method
To synchronize measurements of devices using these functions, as a rule, GPS time is used using substation synchronization equipment (substation clocks).
IEC 61850-90-5 presents the requirements for event timing measurements and measurement timing in five time classes ranging from 1 ms to 1 µs (see TABLE 1 and TABLE 2).
Table 1: Time classes for event measurement according to IEC6185090‑5
|
Time class |
Accuracy |
Measurement |
|
± 1 ms |
Event timestamps |
|
|
± 100 µs |
Zero crossing and data marks for timing control. Equipment switching with synchronism control |
Table 2: Time classes for synchronizing data from instrument transformers according to IEC6185090‑5
|
Time class |
Accuracy |
|
± 25 µs |
|
|
± 4 µs |
|
|
± 1 µs |
Traditional synchronization methods
Depending on the requirements and measurements performed, several methods are used to transmit synchronization signals from the substation synchronization equipment to the involved devices. Most traditional methods require a separate time signal circuit, as shown in FIGURE 1.
Rice. 1: Functional diagram of synchronization signal transmission through a separate distribution channel
To transmit a synchronization signal at substations, the following main methods are used:
IRIG-B. Encryption using the IRIG (Inter Range Instrumentation Group) Time Codes method was developed by the US military department to standardize measurements obtained from sources having different locations. Currently, IRIG-B encryption is primarily used in civilian applications, including power generation facilities. IRIG-B transmits a synchronization signal at 100 bps; Moreover, depending on the transmission method (Unmodulated code (0/+5 V offset) or modulated code (1 kHz carrier)), synchronization accuracy from 1 ms to 10 μs is possible. IRIG-B uses twisted pair or coaxial cable to transmit the signal.
1 pulse per second (1 PPS). The 1 PPS digital signal has wide application as a synchronization signal and is used in many substations. The signal is a regular rectangular pulse with a frequency of 1 Hz, in which a leading or falling edge indicates the beginning of a second. The synchronization accuracy when using such a pulse is on the order of several nanoseconds. Taking into account the signal transmission delay in the physical transmission channel, the achieved accuracy with this method can be 1 μs. The 1 PPS signal itself does not contain additional time information, so the edge of the pulse can be tied to a specific absolute time. As a result, additional timing information must be transmitted to the MFP using a separate auxiliary timing system (eg NTP). In this regard, the 1 PPS method has recently lost its relevance for synchronization purposes at power facilities.
Serial signal transmission in ASCII. This synchronization method is presented for the purpose of completeness of presentation of the material. Due to the availability of better alternatives, this method is very rarely used in the electric power industry. In such schemes, the clock signal is transmitted over a serial transmission channel in ASCII format. The accuracy of synchronization is very dependent on the transmission speed and the quality of the equipment and software. At speeds of 19200 baud and above, accuracy of up to 1 ms is typically achieved.
Rice. 2: Functional diagram of synchronization signal transmission over the station network
As mentioned above, the number of applications of Ethernet networks in substations is steadily growing. In this regard, the relevance of using synchronization systems based on such networks increases (see examples in FIGURES 1 and 2). Prior to the development of the IEEE 1588-2008 standard, NTP was the only widely used synchronization method that did not require a separate time signal transmission system over the network.
NTP. The Network Time Protocol (NTP) is used to synchronize time across computer networks and is primarily designed for reliable synchronization across varying packet rates on networks such as the Internet. The accuracy of synchronization directly depends on network traffic and delays in operating systems. To estimate the average delays during NTP transmission from a server to an individual network client, special calculation algorithms are used. In an Internet environment, accuracy typically reaches 10 ms. In networks used at power facilities, accuracy on the order of several milliseconds can be achieved. This accuracy is sufficient to set a specific absolute time for the rising edge of the 1 PPS signal. However, such a scheme is rarely used due to the fact that two separate reference time transmission channels are required (for example, NTP and 1 PPS).
Table 3: Main characteristics of traditional synchronization methods
|
System |
Accuracytransfers |
Separate transmission channel |
Uncertainty |
|
IRIG-B |
from 10 µs to |
1 year |
|
|
1PPS |
1 µs |
1 second |
|
|
Afterbirth. ASCII |
1 ms |
absent |
|
|
from 1 ms to |
absent |
TABLE 3 summarizes the main characteristics of traditional time synchronization methods. From the experience of using these types of systems, the following requirements can be deduced that an improved time synchronization system must have:
- High timing accuracy
- (1 µs or higher)
- Possibility of using an existing Ethernet network in the system smartgrid
- Automatic compensation of signal propagation delays
- No uncertainty
- Possibility of system redundancy
PRECISION TIME PROTOCOL (PTP)
The IEEE 1588-2008 standard time protocol provides synchronization in information networks, for example, using Ethernet. As with NTP, a common cable is used for basic data transmission and synchronization signal transmission, which allows the use of an already existing information network. Unlike systems with separate channels for transmission, the delay times in this case cannot be calculated based on the cable length. The speed of data packets passing through an information network can change dynamically, and the network infrastructure can change, so the transmission delay of each data packet must also be adjusted dynamically. Network switches can also introduce additional delay when transmitting packets, and this delay can be much greater than the delay due to cable length. The precise time protocol takes into account the dynamic nature of changes in delays during signal transmission and allows you to automatically make the necessary adjustments.
Synchronization via PTP protocol
The principle of operation of the method is shown in FIGURE 3. The master synchronization device Master (for example, a GPS synchronizer) and the slave device Slave (for example, a relay protection device) are connected via an information network. The task is to synchronize the Slave and Master devices in such a way that both of them provide the same time synchronously.
The time deviation between devices is expressed by Δ t ms. FIGURE 3 also shows this deviation as shifted zeros on the time axis. The goal is to measure the value Δ t ms. To do this, data packet A is sent from the Master device via the Ethernet network to the Slave device. In this case, the device M determines the moment in time t 1 when the package was sent. So, time t 1 this is the absolute value of the time that the data packet was sent from the clock master. The data packet reaches the Slave device over the network after a certain time. This delay is designated as Δ tp in FIGURE 3 and is the sum of all signal delays in the cable and network switches. After this delay, the data packet reaches the Slave device, in which the next time stamp is generated, designated as t 2".
Rice. 3: Determining the signal transit time between devices when transmitting 2 data packets in opposite directions
Thus, the Slave device determines the time when data packet A reaches it. In this case, the relationship between times t 1 And t 2" determined by the formula:
t 2 "= t 1 + Δ tp - Δ t ms (1)
Following this, the Slave device sends a data packet B to the Master device. Time the packet was sent by the Slave device ( t 3") and the time the packet was received by the Master device ( t 4) is remembered for further calculation. If the physical channel of the packets is the same, we can assume that the delay Δ tp will be exactly the same and the final time will be equal to:
t 4 = t 3" + Δ tp + Δ t ms (2)
Time values are present in both devices: t 1 And t 4 in the Master device, t 2" And t 3" in the Slave device. As soon as the Master device transmits its values ( t 1 And t 4) to the Slave device in the data packet, the Slave device can solve the system of equations (1) and (2) and thus find the time value Δ t ms according to the formula:
Δ t ms = (t 1 - t 2"- t 3" + t 4) / 2 (3)
As a result, the Slave device can use this information to adjust its time. Constant repetition of this measurement (usually 1 time per second) and subsequent correction of the signal transit time between devices allows reducing the time deviation to 100 ns.
For the considered delay measurement, it is extremely important that the transit times of packets A and B are the same, i.e. did not depend on the direction of packet transmission. This requirement is not met in standard network topologies, due to the fact that Ethernet switches store incoming packets for some time before sending them. This residence time (the time a packet is stored in the switch before being sent) of a packet in the switch depends on a number of factors (for example, traffic load) and can lead to inaccuracies. To solve this problem, the IEEE 1588-2008 standard uses the open synchronizer (OT) method. Such a synchronizer is a switch that measures the time it takes for a PTP message to travel and transmits this information to devices that receive the corresponding PTP messages.
Mechanisms for measuring delays
Based on the principle stated above, the IEEE 1588-2008 standard proposes two mechanisms for measuring delays: end-to-end (E2E) and peer-to-peer (P2P). For secure synchronization, all devices on the same network must use the same measurement mechanism.
End-to-end delay measurement mechanism. When using the E2E mechanism, the system's open synchronizer measures the residence time of the PTP message and records this information in the event correction field. If a PTP message passes through several open synchronizers, all the residence times of this message in the synchronizers are accumulated in the correction field.
Peer-to-peer latency measurement mechanism. If in the E2E mechanism open synchronizers measure only the message dwell time, then in the P2P mechanism open synchronizers also measure the delay between the message receiving and transmitting ports. As a result, the PTP message correction field will contain the residence time in all open synchronizers and the delay time between links in the transmission channel.
In an E2E configuration, latency measurements occur individually between the Master device and each Slave device connected to it, as shown in FIGURE 4. As a result, traffic towards the device Master will be increased because Master sees all devices connected to it.
Rice. 4: Ring topology for end-to-end delay measurements
In P 2 P configuration open synchronizers measure signal delays between adjacent synchronizers, as shown in FIGURE 5.
Rice. 5: Ring topology for peer-to-peer latency measurement
Latency measurements are also performed on transmission segments that are blocked by redundancy protocols (eg, Rapid spanning tree). In this way, synchronization-safe reconfiguration is possible, since when changing the synchronization network, there will be no need to recalculate time delays.
Algorithm for the best synchronizer
Another feature of the protocol described in the IEEE 1588-2008 standard is the Best Master Clock Algorithm (BMCA). This algorithm automatically allows you to determine the most efficient synchronizer, which is subsequently used as the main one for the entire network. This synchronizer becomes the leading one and all other synchronizers adjust their time to it. This way there is no need to manually select the master network synchronizer. Best Master Clock Algorithm also includes a redundancy function. If the leading synchronizer is not functioning, the next most efficient synchronizer becomes the leading synchronizer automatically. For complex network infrastructures, the protocol provides the function of defining a master synchronizer that will be used for further synchronization.
Rice. 6: Best Master Clock Algorithm (BMCA) ) in a system of two subsystems with six synchronizers ( C 1… C 6).
FIGURE 6 shows a network consisting of 6 synchronizers (C1...C6) connected through 2 switches (S1 and S2). The C4 synchronizer is the best in terms of characteristics, because has a GPS receiver and therefore can receive a high-precision signal from the satellite. Due to the fact that this synchronizer provides the highest accuracy, the BMCA algorithm sets synchronizer C4 as the master for the entire network. All other devices in the network (C1, C2 ... which may be part of protection terminals, etc.) are synchronized relative to the time of device C4.
C3 operates in a special mode. This device has 2 ports, so it can connect two networks together through switches S1 and S2. According to the BMCA algorithm, the network port of device C3 on the side of switch S1 is configured as a slave (denoted by the letter S in FIGURE 6) and is adjusted to the time of master device C4. For a system operating through switch S2, synchronizer C3 becomes the master and transmits the time received from C4 to devices C5 and C6. In IEEE 1588-2008 terminology, the C3 device is called a Boundary Clock. Such a device allows you to synchronize the time of two isolated networks relative to a common reference time. This configuration is automatically provided by the BMCA algorithm. When you disconnect a device or add a new device, the system will be reconfigured automatically.
PTP Profiles
IEEE 1588-2008 is a fairly complex standard that allows you to set custom settings for various applications where the PTP protocol may be used. To ensure flexible operation and quick configuration of equipment operating via PTP, the standard includes preset configuration profiles. Profiles define default settings as well as synchronization types depending on the application. Annex J of IEEE 1588-2008 defines two default profiles: the Request-Response Default PTP profile (or E2E profile) and the Peer-to-Peer Default PTP profile. Interested organizations (standardization, industrial committees, etc.) have the opportunity to create additional profiles.
PTP POWER PROFILE
For safe operation of equipment according to the protocol PTP in the field of electric power industry standard IEEE C 37.238-2011 defines the so-called Power Profile . This profile was created by working groups WG H 7 of the IEEE Power Systems Relaying Committee and WG C 7 of the Power Systems Substation Committee . Both groups work under the auspices of the community IEEE Power and Energy Society.
This profile, defined in IEEE 1588-2008, created to ensure time synchronization in critical nodes of the power system operating 24 hours a day. For this purpose, in addition to the selected parameters IEEE 1588-2008 specific profile system parameters were defined.
IEEE 1588-2008 PTP Power Profile parameters
This section describes the main IEEE 1588-2008 parameters that are used in the PTP power profile. A full description of the parameters is given in the standard.
Synchronizer type. In the PTP power profile, you can select a one-step synchronizer (A one-step synchronizer places a timestamp directly in the PTP message (eg Sync). A two-step synchronizer sends the timestamp in a separate Follow_Up message) or a two-step synchronizer. It is recommended to use a single-step type of synchronizer, which is more modern. Two-step synchronizers were included in the profile only due to their availability in the industry. It is also possible to select all types of synchronizers described in IEEE -1588-2008 (simple, open, edge).
Algorithm for the best synchronizer (BestMasterClockAlgorithm). The PTP power profile uses the BMCA algorithm. The main feature of the BMCA setup is that only potential master synchronizers (A potential master synchronizer has a high-precision reference signal (eg a GPS system)) have the ability to act as candidate masters. All other simple synchronizers operate in slave-only mode. Thus, only a potential master synchronizer associated with an external reference signal can be a substation master.
Peer-to-peer delay mechanism. The PTP power profile specifies a peering mechanism for transmission delays. The advantage of this setup is that pre-measuring all delays between nodes allows you to quickly adjust the transmission when measuring the network configuration, and the load on the synchronization master is significantly reduced.
TLV-local time tag. In the PTP power profile, potential master synchronizers must add a TLV to their identification messages (Identification messages are defined in IEEE 1588‑2008 and contain information about the synchronizer (eg, synchronizer quality tags, identification tags, etc.). TLV indicator contains time zone information and other information necessary for the device to convert UTC time to local time.
PTP Power Profile Specific Parameters
This section describes additional parameters that must be set to integrate devices according to the standard IEEE 1588-2008 in substations using IEC 61850 , :
Ensuring sustainable operation. To ensure the required time accuracy even in the most complex applications (synchrophasors, sampled values), parameters are set for the stable transmission of time synchronization signals to slave devices (e.g. protection terminals).
The total error at the input of the slave synchronizer should not exceed 1 μs after 16 retransmissions. As shown in FIGURE 7, the master synchronizer allows a maximum error of no more than 200 ns, and open synchronizers can introduce an additional error of no more than 50 ns. This stability of operation is determined for 80% network load. To achieve stable operation within such limits, open synchronizers must be at least syntonized (Devices are syntonized if the duration of a second is the same in them. The sampling periods of devices may differ).
Rice. 7: U conditions for ensuring sustainable work on IEEE C 37.238-2011
Time to transfer the master synchronizer function to another device. In IEEE standard 1588-2008 the time shift is not defined when transferring the master synchronizer function from one device to another; in profile PTP power profile a maximum shift of 2 μs is specified for 5 s at a constant temperature. This means that if synchronization is lost, the master synchronizer should not move more than 2 µs in 5 s. This period is considered necessary to ensure that another device in the system has sufficient time to enter master mode.
IEEE Tags802.1Q. The PTP power profile enforces the requirement that all PTP messages comply with IEEE 802.1Q definitions. Each frame contains a tag that indicates the priority of the frame and the status of the frame in the virtual network VLAN (Virtual Network). The priority field ensures that the most important messages (eg substation protection messages) have the highest priority. VLAN fields allow you to divide a physical network so that messages intended for a specific device are transmitted to that specific device. Using VLANs can improve system security by blocking security threats and ensuring message confidentiality. As a result, the total network traffic also decreases.
IEEE Control BaseC37.238. The PTP power profile specifies the control base Management Information Base (MIB) for the record Simple Network Management Protocol (SNMP). SNMP traps are included in the MIB to indicate event changes (eg master clock change).
TagsTLVIEEE C37.238. The PTP power profile defines TLV tags that contain information about the master synchronizer, master synchronizer timing error, and network timing error. These parameters can be used by the MFP to determine the most likely time error and, therefore, make it possible to evaluate the quality of the issued time stamps.
IMPLEMENTATION AND SCENARIOS OF TRANSITION TO STANDARD AT NEW SUBSTATIONS
Comprehensive structure of the standard IEEE 1588-2008 allows you to develop various time synchronization concepts for a specific power facility, depending on the customer’s wishes.
Implementation of PTP at substations under construction
If a substation is being built, it is possible to carry out the initial design of the synchronization network according to IEEE 1588-2008, because the entire network infrastructure can take IEC requirements into account in advance 61850 and PTP protocol.
Network infrastructure. The network infrastructure design can be adopted as for a standard substation with IEC61850 All security or network redundancy considerations (eg ring topology) can be taken into account. The only requirement for synchronization hardware is that the network be built on devices that support PTP (=open synchronizers). To ensure interoperability, all devices on the network must be able to work in a profile PTP power profile.
Reservation. To ensure reliable PTP operation, it is recommended that there are 2 or 3 potential master synchronizers in the network. The GPS antennas of these devices should be installed in different locations to minimize the risk of signal loss due to reception problems.
Network security. In general, the same requirements and recommendations as for IEC should apply61850. In addition, it is recommended to use the virtual network function VLAN tags IEEE 802.1 Q profile PTP power profile . To do this, the switches used must support the reception and delivery of traffic with tags IEEE 802.1Q.
Structuring the system's timing parameters. Some functions (eg synchrophasors, sampled values) require accurate timing and synchronization quality information. Synchronizers in the PTP power profile provide such information (see IEEE C 37.238 TLV Tags). PTP power profile Information Annex C describes how timing parameters can be included in IEC61850 functions, such as device timestamps or sampled values.
Because there are so many infrastructure options, there is no single way to set all parameters.
Rice. 8: Example of PTP implementation in a modern substation
FIGURE 8 shows an example implementation for an IEC 61850 substation. For the sake of simplicity, the IEC 61850 station bus and the IEC 61850 process bus are shown using open synchronizers S1 and S2 (In reality, the infrastructure is implemented using several switches). The question of how to implement process and station buses (and whether this should be done at all) has been discussed several times. Solutions can vary: from two completely independent networks to one common network infrastructure. From the point of view of IEEE 1588-2008, all devices at a power facility must be synchronized to one master synchronizer. This means that the station bus and the process bus must be connected in some way. An open switch, a PTP-enabled router, or a boundary synchronizer can serve this purpose, as shown in FIGURE 8. A boundary synchronizer avoids the direct connection between the IEC 61850 station bus and the IEC 61850 process bus if such an architecture is required. The illustrated circuit also provides full redundancy. If synchronizer C1 exceeds the required accuracy, the next device in the system - most likely C2 - becomes the master synchronizer for the entire system.
Transition to standard at existing substations
When implementing IEEE 1588-2008 in newly built substations with IEC 61850 certain difficulties may arise. If the cable connections of the existing network can be used, then the network devices must be replaced with ones that support the protocol PTP commutators (=open synchronizers). However, it is positive that not all network devices need to be replaced at once. Implementation PTP may be limited to areas where high timing accuracy is required. Modern synchronizers PTP can work in parallel using protocols NTP and PTP over the same network. Therefore, areas where less time accuracy is allowed can work according to the protocol NTP.
Integration of devices that do not support PTP can be accomplished in different ways. One method is to provide clock signals locally, as shown in FIGURE 8. Device C 3 synchronizes with the master clock and locally provides clock signals (eg, IRIG-B or 1 PPS) to non-PTP devices. These devices (indicated in gray in FIGURE 8) are connected to the IEC61850 process bus and receive a time synchronization signal from device C 3 via a separate communication channel. Open synchronizers with additional IRIG -B outputs for connecting older device types are already being produced.
U devices that do not support PTP , do not have an interface Ethernet , providing hardware synchronization, which is necessary for accurate work on PTP . But, in principle, in some modern devices it is possible to update the software and hardware, which will allow synchronization by PTP . When using software timestamps, the total accuracy can reach from 20 μs to 100 μs. This is sufficient for the general requirements as defined in the IEC61850-90-5 (see TABLE 1), but not enough for accuracy classes T 3… T 5 (see TABLE 2). Thus, for devices that must provide accuracy classes from T 3 to T 5, it is necessary to be able to upgrade before using hardware synchronization or they need to be replaced with new devices. Due to the fact that many substations where time synchronization is required currently use IRIG-B, in IEEE C standard 37.238-2011 (Appendices C and D ) provides retrofitting guidelines for application PTP power profile at such facilities.
Delay in antenna cable
It was shown above that when implementing PTP, all delays introduced by the network and network equipment can be automatically compensated. All that remains is the need to manually compensate for the delay introduced by the cable between the GPS antenna and the master synchronizer. Moreover, even special high-frequency cables have a fairly high attenuation at the GPS reception frequency (1.57542 GHz), which limits the maximum cable length to 50 to 100 m. If, due to difficult signal reception or special operating conditions (for example, the station is located in a mine), it is necessary to cover a long distance for transmission, additional measures must be taken (signal amplifiers, use of intermediate frequency).
Rice. 9: Master synchronizer built into the antenna
If the PTP master synchronizer is built into the antenna (see FIGURE 9), coaxial cable between the synchronizer and the antenna will not be required. Communication between the devices and the master synchronizer is implemented via an Ethernet network. The master synchronizer can also be powered via an Ethernet cable using Power over Ethernet (PoE) technology. When using a standard Ethernet cable, signal transmission over a distance of up to 100m is possible. If an optical Ethernet channel is used, the distance between an external antenna with a built-in master synchronizer and synchronized devices in the network can be increased to 2 kilometers. In this case, there is no need to compensate for the delay in the antenna cable, due to its absence.
CURRENT ISSUES AND PERSPECTIVES
The IEEE 1588-2008 standard with the PTP power profile according to IEEE C 37.238 offers a comprehensive solution for implementing accurate time synchronization over an Ethernet network. In this case, the questions that arise, as a rule, relate to general problems that are not directly related to IEEE 1588-2008 or are indirectly related to the standard.
One of these general problems within the framework of the issue of time synchronization is the issue of reliability of the GPS system. Currently, GPS is the only power system standard in use that provides sub-microsecond accuracy. A possible solution is to use backup systems based on other standards and technologies (GLONASS or highly stable oscillators to compensate for GPS signal losses).
Another issue is the security of the information network in general. In this regard, it is recommended to use the transmission path selection procedure (circuit decoupling) using IEEE 802.1Q tags in accordance with the PTP power profile. This provision is also consistent with the security standard IEC 62351-6 (Part 4.1), which recommends using path selection instead of encryption for synchronization-critical processes.
Due to the growing interest in the IEEE 1588-2008 standard in networks using Ethernet, there is now a wide variety of network devices and synchronizers that can be used in this environment. PTP integration has already been implemented for many existing devices or is planned by leading manufacturers. Thus, consideration of IEEE 1588‑2008 can be recommended for use in new substations or in the construction of planned facilities.
CONCLUSION
The IEEE 1588 standard consistently defines all the steps to provide a reliable, secure, and easy-to-use time synchronization system. In this case, a separate communication channel via cable is not required, i.e. costs are reduced and the organization of a separate time synchronization network is not required. The IEEE C 37.238-2011 PTP power profile provides full integration of IEEE 1588-2008 into a system using IEC61850. Thus, there is every reason to believe that the precise time protocol is an optimal and flexible method for ensuring time synchronization at modern power generation facilities.
LITERATURE
1. IEEE 1588-2008, “IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems”, IEEE, 2008
2. IEEE C37.238-2011, “IEEE Standard Profile for Use of IEEE 1588 Precision Time Protocol in Power System Applications”, IEEE, 2011
3. IEC 61850 Ed.2, “Communication networks and systems in substances”, IEC
4. RFC 5905, “Network Time Protocol Version 4: Protocol and Algorithms Specification”, Internet Engineering Task Force (IETF), 2010
5. IRIG Standard 200-04, “IRIG Serial Time Code Formats.” Range Commanders Council, 2004
6. Baumgartner B, Riesch C, Rudigier M, “IEEE 1588/PTP: The Future of Time Synchronization in the Electric Power Industry”, PAC World Conference 2012, Budapest, Hungary, 2012
7. PRC-018-1, “Disturbance Monitoring Equipment Installation and Data Reporting” NERC, 2006
8. Dickson B,”Substance time Synchronisation” PAC World Magazine, Summer 2007 Issue, 2007
9. Weibel H, “Technology Update on IEEE 1588 - The Second Edition of the High Precision Clock Synchronization Protocol”, Embedded World 2009, Nürnberg, Germany, 2009
10. Antonova G, “Standard Profile for Use of IEEE Std 1588-2008 Precision Time Protocol (PTP) in Power System Applications”, PAC World Conference 2012, Budapest, Hungary, 2012
11. IEEE 802.1Q-2011, “Media Access Control (MAC) Bridges and Virtual Bridge Local Area Networks”, IEEE, 2011
12. Eidson J C, “Measurement, Control and Communication Using IEEE 1588.”, Springer-Verlag, London, 2006
13. Steinhauser F, Riesch C, Rudigier M: “IEEE 1588 for time synchronization of devices in the electric power industry”, ISPCS 2010; Portsmouth, NH, USA, 2010
14. IEC 62531-6, “Power systems management and associated information exchange -Data and communications security -Part 6: Security for IEC 61850”, IEC, 2012
Why do you need exact time?
Who needs this exact time anyway? Of course, we, the users, need it so that we are less late. Let's imagine a modern airport - for its operation, hundreds of pilots and dispatchers must use an unerringly running clock. The system for registering goods in warehouses, hospitals, railway ticket offices and many other institutions require that the time at all objects of the system be the same to one degree or another. Especially computers. They run a lot of services and programs, the normal operation of which requires precise time, and, as a rule, more precise time than we humans usually need. System services, security system components, and simply application programs can be very critical to the accuracy of the clock. The most prominent example of such services is the Kerberos authentication protocol. So, for it to work, it is necessary that on computers accessed using this protocol, the system time differs by no more than 5 minutes. In addition, accurate time on all computers greatly facilitates the analysis of security logs when investigating incidents on the local network.
NTP protocol
NTP (Network Time Protocol) is a protocol designed to synchronize time on a network. It is a set of fairly complex algorithms designed to ensure high accuracy (up to several microseconds) and fault tolerance of the time synchronization system. Thus, the protocol involves simultaneous synchronization with several servers.
There are several versions of this protocol, with some differences. The third version of this protocol was standardized as RFC 1305 in 1992. The fourth (currently latest) version brings some improvements (automatic configuration and authentication, improved synchronization algorithms) compared to the third, but it is not yet standardized in the RFC.
In addition, in addition to the NTP protocol, there is the SNTP (Simple Network Time Protocol). At the packet level, the two protocols are fully compatible. The main difference between them is that SNTP does not have the complex filtering systems and multi-stage error correction found in NTP. Thus, SNTP is a simplified and easier to implement version of NTP. It is intended for use in networks where very high time accuracy is not required, and in Microsoft's implementation it provides accuracy within 20 seconds within an enterprise and no more than 2 seconds within a single site. The SNTP protocol is standardized as RFC 1769 (version 3) and RFC 2030 (version 4).
The NTP synchronization model assumes a hierarchical structure. At the first level of the hierarchy there are so-called “primary” time servers (First stratum). They are located in different places around the world and have the most accurate time. There are relatively few such servers, since the exact time on them is maintained using expensive specialized equipment (radio channel, satellite channel). Second-level servers (Second stratum) are synchronized with first-level servers using the NTP protocol. There are already significantly more of them, but they are already somewhat out of sync (from 1 to 20 milliseconds) relative to the “primary” servers. Next can be servers of the third, fourth and subsequent levels:
With the transition to each level, the error relative to the primary server increases slightly, but the total number of servers increases and, consequently, their load decreases. Therefore, as an external source of synchronization, instead of using primary servers that have the most accurate time, it is recommended to use secondary servers as they are less loaded.
To synchronize time in Windows 2000/XP/2003, the SNTP protocol is used. Support for this protocol is implemented in the form of the Windows Time system service, which is part of the MS Windows 2000/XP/2003 operating system. A distinctive feature of this implementation is that the Windows Time service supports domain authentication when accessing the reference time server, which is important from a security point of view.
There are several options for operating the SNTP service included in Windows:
- Hierarchical (NT5DS). Used by default for all computers joined to a domain. Time synchronization on workstations and domain servers is carried out in a hierarchy. Thus, workstations and member servers are synchronized with the domain controller that authenticated the login, domain controllers are synchronized with the master of the “PDC emulator” operation, which in turn is synchronized with the domain controller located at a higher level of the hierarchy. It should be noted that this synchronization order is used “by default” and can be overridden manually or using group policies. How to do this will be discussed below.
- Force synchronization with the selected NTP server (NTP). In this case, the reference time source for the Windows Time service is set either manually or using group policies.
- Disable synchronization (NoSync). This mode is required for a mixed time management scheme, in which a third-party product is used to synchronize with an external source, and Windows Time is used to maintain time within the domain.
Thus, in the case of a workgroup, time synchronization will still have to be configured manually. For example, one of the computers can be configured to synchronize with an external server using the SNTP protocol, and the rest can be configured to synchronize with it. The steps required for this will be described below.
For a domain, it is recommended to use hierarchical synchronization using the SNTP protocol. In most cases, it provides acceptable system time accuracy within the domain's forest. In addition, it automatically ensures time update security by supporting Active Directory authentication. To maintain the “correct” time in the domain, it is necessary to synchronize the top-level domain controller, which owns the “PDC emulator” role, with an external NTP server. In our example, the role of such a server will be a Linux machine with the ntpd daemon running.
Setting up SNTP on Windows
To configure the Windows Time service, two utilities are used:
- Net time
- W32tm
Net time is used primarily for configuring the time service, and w32tm is used for monitoring and diagnosing operation. However, in Windows XP/2003, the w32tm utility has undergone significant changes and can be used to configure the time service. NTP configuration will be further carried out using Windows XP/2003 as an example.
So, in order to “manually” specify the synchronization source using net time, just write on the command line:
et time /setsntp:list_of_time_servers_separated by_space
To obtain information about the current time server:
net time /querysntp
You can find out the time on a domain controller like this:
net time /domain:domain_name
And synchronize time with a domain controller like this:
Net time /domain:domain_name /set
The w32tm system utility can do all the same and even more:
- w32tm /resync - Using this command, you can force a local or remote computer to synchronize its system clock with the time server it uses.
- w32tm /config – This command is used to configure the Windows Time service. With its help, you can specify the list of time servers used and the type of synchronization (hierarchical or selected by servers).
For example, to override the default values and set up time synchronization with an external source, you can use the command:
w32tm /config /syncfromflags:manual /manualpeerlist:PeerList
And in order for Windows Time to apply the new settings, instead of restarting the service, you can use the command:
w32tm /config /update
In addition, w32tm provides the following options related to time monitoring on computers:
- w32tm /monitor – using this option you can find out how the system time of this computer differs from the time on the domain controller or other computers.
- w32tm /stripchart – graphically shows the time difference between the current and remote computer.
- w32tm /register – Registers the Windows Time service as a service on this computer. This option can be useful on computers that are not part of a domain, since by default the time service is stopped on them.
More detailed information about the parameters of the net time and w32tm utilities can be obtained using the /? key. or by opening the appropriate section of the Help and Support Center MS Windows XP/2003 help system.
It is not difficult to guess that the Windows Time service settings are stored in the Windows registry in the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\ section.
In the root of the section, the operating parameters of the service itself are defined, in the Config subkey – settings related to the operation of the SNTP protocol itself, the synchronization mode is defined in the Parameters subkey. The SNTP client and server settings are located in the TimeProviders\NtpClient and TimeProviders\NtpServer connections, respectively. Let's look at the main values that determine the NTP client and server settings:
- Type – determines the operating mode of the NTP client (NTDS5 – hierarchical, NTP – “manual”, NoSync – do not synchronize, AllSync – all types of synchronization are available);
- Enabled – determines whether this component (client or server) is enabled;
- CrossSiteSyncFlags – determines whether it is possible to synchronize time with a source located outside the domain if hierarchical synchronization is used (0 – not possible, 1 – only with the PDC emulator, 2 – with all);
- EventLogFlags – determines whether messages from Windows Time will be logged or not (a very useful feature when debugging).
Another option for configuring the Windows Time service is to use Group Policies. The settings are defined in the Group Policy Object at the following address: “Computer Configuration –> Administrative Templates –> System –> Windows Time Service”.
If you have Windows 2000 Server installed and you haven’t found such a setting, don’t despair, you just need to update the “Administrative Templates”. To do this, copy all .adm files from the system32\GroupPolicy\Adm system folder of any machine with Windows XP installed to the server that is a domain controller. Next, while defining the GPO, right-click on “Administrative templates” and select “Add/Remove templates...” Remove the templates listed there and add the copied ones. After clicking the "OK" button, the templates will be updated and you can configure the time service using group policies:
It is easy to see that this mainly lists all the settings that can be changed in the registry. There is nothing surprising in this, because this is exactly how most group policies work.
In Windows XP, another way to set a time server has appeared, which can be very convenient for setting up synchronization on a home computer or a computer that is part of a workgroup:
NTP server for Linux - external synchronization for Windows domain
As mentioned above, the NTP protocol is more error-resistant, so it is better to use an NTP server as a source of reference time for external synchronization. In addition, the top-level domain controller does not always have access to the Internet via UDP port 123, which is used for NTP operation. Access may well be denied for security reasons, which is common practice for large organizations. In such cases, to solve this problem, you can install your own time server in the DMZ, configured to synchronize with an external source, and use it as a reference time source for synchronizing the top-level domain controller. Any computer, not necessarily a modern one, with a *nix-like OS, for example, Linux, installed in a minimal configuration, without an X server and other potentially vulnerable things, is quite suitable as such a computer.
There are a lot of time synchronization programs for Linux OS. The most famous are Xntpd (NTP version 3), ntpd (NTP version 4), Crony and ClockSpeed. In our example, we will use the ntpd ntp server included in Redhat 9, supplied in the ntp-4.1.2-0.rc1.2.i386.rpm package.
The package includes several programs designed to work with NTP.
Here are the main ones:
- Ntpd – ntp daemon that maintains accurate time in the background;
- Ntpq is a utility designed to poll NTP servers that support the standard polling protocol NTP mode 6. With its help, you can find out and change the current state of the server, if its settings allow it;
- Ntptdc – a utility with which you can interrogate the ntpd daemon and obtain statistics on its operation;
- Ntpdate is a program for setting the current system time using the NTP protocol.
A standard feature of the NTP protocol is the ability to perform authentication. In this case, both symmetric algorithms (DES), which appeared in the second version of the protocol, and asymmetric algorithms with a public key, which are an innovation in the fourth version, can be used.
In the case of using a symmetric authentication scheme, the client and server select an arbitrary identifier and one of 65534 keys defined by the standard. When using asymmetric algorithms, the so-called Autokey scheme is used, the distinctive feature of which is that there is no need to pre-distribute server public keys.
To configure authentication in ntpd, there are utilities ntp-genkeys, ntpq and ntpdc.
All NTP functionality related to accurate time support is implemented in the ntpd daemon. It is configured in the usual UNIX way - by editing the ntp.conf configuration file located in the /etc folder.
Let's set the following options for the NTP server.
First, let's indicate the servers with which time synchronization will be performed:
server ntp.nasa.gov # A stratum 1 server at nasa.org
server ntp1.demos.net # A stratum 2 server at demos.net
restrict ntp.research.gov mask 255.255.255.255 nomodify notrap noquery
Here the mask 255.255.255.255 is used to restrict access to our server from the ntp.nasa.gov server. Now let's define a list of hosts on our local network that we want to allow access to our NTP server to get the time:
restrict 192.168.1.0 mask 255.255.255.0 notrust nomodify notrap
We also require that the Linux machine has full access to its server resources:
restrict 127.0.0.1
And now the most important thing: we need to make sure that the default deny, which has a higher priority, is commented out:
#restrict default ignore
After saving the ntp.conf file, the configuration can be considered complete, but it may happen that after starting the daemon, the time will still not be synchronized. The fact is that the NTP protocol was originally developed as a protocol for maintaining time, not setting it. Therefore, if the difference between the clock readings is large enough (more than a few minutes), then synchronization will not occur. In this case, it makes sense to initially set the time manually using the ntpdate command (you can also add the ntpdate command to the machine’s startup scripts):
# ntpdate clock.vsu.ru
17 Feb 23:44:54 ntpdate: step time server 198.123.30.132 offset 25.307598 sec
17 Feb 23:45:05 ntpdate: adjust time server 198.123.30.132 offset 0.002181 sec
# ntpdate clock.vsu.ru
The ntp daemon is launched through initialization scripts. If the program was installed from an rpm package, then most likely all issues related to its automatic launch have already been resolved. To verify this, you can use the command:
# chkconfig -list ntpd
ntpd 0:on 1:off 2:off 3:on 4:off 5:on 6:off
If you do not see this line, it means that ntpd is not configured to start automatically. To fix this, type:
# chkconfig –level 035 ntpd on
To manage NTP (start, launch, restart, status), a standard initialization script is used:
# /etc/init.d/ntpd start
To view server synchronization statistics, you can use the following command:
# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*tick.usnogps.na .USNO. 1 u 38 64 377 220.00 0.149 7.08
-navobs1.wustl.e.PSC. 1 u 55 64 377 193.47 6.857 4.81
-ntp-nasa.arc.na .GPS. 1 u 43 64 377 254.88 7.471 9.58
-ntp0.NL.net .GPS. 1 u 144 512 377 122.54 12.515 13.75
-ntp2.ien.it .IEN. 1 u 558 1024 377 133.94 14.594 41.99
+timekeeper.isi. .GPS. 1 u 13 64 377 245.30 3.454 15.08
+news-zero.demos ntp0.usno.navy. 2 u 16 64 377 37.51 -3.239 11.14
LOCAL(0) LOCAL(0) 10 l - 64 377 0.00 0.000 10.01
NTP server/client operating modes
Client/server
This mode is by far the most commonly used on the Internet. The work scheme is classic. The client sends a request, to which the server sends a response within some time. The client is configured using the server directive in the configuration file, where the DNS name of the time server is specified.
Symmetrical active/passive mode
This mode is used if time synchronization is performed between a large number of peer machines. In addition to the fact that each machine synchronizes with an external source, it also synchronizes with its neighbors (peers), acting as a client and time server for them. So even if a machine "loses" an external source, it will still be able to obtain accurate time from its neighbors. Neighbors can work in two modes – active and passive. Working in active mode, the machine itself transmits its time to all neighboring machines listed in the peers section of the ntp.conf configuration file. If neighbors are not indicated in this section, then the machine is considered to be operating in passive mode. To prevent an attacker from compromising other machines by posing as an active source, authentication must be used.
Broadcast Mode
This mode is recommended for use in cases where a small number of servers serve a large number of clients. When operating in this mode, the server periodically sends packets using the subnet's broadcast address. A client configured to synchronize in this manner receives the server's broadcast packet and synchronizes with the server. A feature of this mode is that time is delivered within one subnet (limiting broadcast packets). In addition, authentication must be used to protect against attackers.
Multicast mode
This mode is in many ways similar to broadcast. The difference is that multicast addresses of class D networks of the IP address space are used to deliver packets. For clients and servers, the address of the multicast group is specified, which they use for time synchronization. This makes it possible to synchronize groups of machines located on different subnets, provided that the routers connecting them support the IGMP protocol and are configured to transmit multicast traffic.
Manycast mode
This mode is an innovation in the fourth version of the NTP protocol. It involves the client searching for manycast servers among its network neighbors, receiving time samples from each of them (using cryptography) and, based on this data, selecting the three “best” manycast servers with which the client will synchronize. If one of the servers fails, the client automatically updates its list.
To transmit time samples, clients and servers operating in multicast mode use multicast group addresses (class D networks). Clients and servers using the same address form the same association. The number of associations is determined by the number of multicast addresses used.
Frequently asked questions
Why is the time not synchronized after the command net time /setsntp:server?
Make sure that the w32time service's startup type is set to Automatic.
Make sure that UDP port 123 of the NTP server you are using is available.
Make sure that the time between client and server does not vary too much.
Can an SNTP client synchronize with an NTP server?
Yes, it can, since the SNTP protocol is a subset of NTP and is fully compatible with it.
Can I use a third party NTP server on Windows 2000/XP/2003?
Yes, you can use any server, paid or free. You must first disable the appropriate components (client or server) of the Windows Time services.
Why does the NTP client not work on a computer with MS SQL Server installed?
Most likely the problem is that SQL Server is somehow occupying UDP port 123. A solution might be to run the NTP client before MS SQL Server.
The ntpd daemon, after starting, runs for 10-20 minutes, after which it stops. What could be the problem?
Make sure that the client and server times do not differ too much (no more than 5 minutes). Otherwise, force synchronization using ntpdate.
Is it possible to synchronize time in OS Windows NT4, 95, 98, Me?
It is possible using third-party programs, for example, NetTime, Automacahron, World Time5, ntpd Windows NT port.
When trying to log into a Windows domain, a message appears that the time between the domain controller and the workstation differs too much, despite the fact that synchronization is precisely configured.
Most likely the problem is that the time has gone very wrong (CMOS reset, hacker sabotage) and the service is unable to authenticate using the Kerberos protocol. To solve this problem, you need to either manually set the time or not use this type of authentication when updating the time.
Many articles have been written about the well-known Network Time Protocol (NTP), some of them mention the Precision Time Protocol, which supposedly makes it possible to achieve time synchronization accuracy of the order of nanoseconds (for example, and ). Let's figure out what this protocol is and how such accuracy is achieved. We’ll also see the results of my work with this protocol.
Introduction
"Precision Time Protocol" is described by the IEEE 1588 standard. There are 2 versions of the standard. The first version was released in 2002, then the standard was revised in 2008 and the PTPv2 protocol was released. Backward compatibility has not been maintained.I'm working with the second version of the protocol, it has many improvements over the first (accuracy, stability, as the wiki tells us). I will not make comparisons with NTP, the mere mention of synchronization accuracy, and the accuracy of PTP actually reaches tens of nanoseconds with “hardware” support, indicates an advantage over NTP.
Hardware support for the protocol can be implemented differently in different devices. In fact, the minimum required to implement PTP is the ability of the hardware to put a time stamp on the moment the message was received on the port. The entered time will be used to calculate the error.
Why does the clock get upset?
Errors can come from anywhere. Let's start with the fact that the frequency generators in the devices are different and there is a very low probability that two different devices will work perfectly in time. This can also be attributed to constantly changing environmental conditions affecting the generated frequency.What are we trying to achieve?
Let's say we have a device that operates under ideal conditions, some kind of atomic clock that will not move at all until the end of the world (of course, before the real one, and not the one predicted by the Mayan calendar) and we are given the task of obtaining at least approximately (with an accuracy of 10 -9 sec) same hours. We need to synchronize these clocks. To do this, you can implement the PTP protocol.The difference between a purely software implementation and an implementation with “hardware support”
A purely software implementation will not achieve the promised accuracy. The time elapsed from the moment of receiving a message (more precisely, receiving a signal to receive a message in the device) until the transition to the interrupt entry point or callback cannot be strictly defined. “Smart hardware” with PTP support can set these timestamps independently (for example, chips from Micrel, I’m writing a driver for the KSZ8463MLI).In addition to time stamps, “hardware” support also includes the ability to tune a quartz oscillator (to align the frequency with the master), or the ability to adjust the clock (increase the clock value by X ns each clock cycle). More on this below.
Let's move on to the IEEE 1588 standard
The standard is described on as many as 289 pages. Let's consider the minimum required to implement the protocol. PTP is a client-server synchronization protocol, i.e. At least 2 devices are required to implement the protocol. So, the Master device is an atomic clock, and the Slave device is a clock that must be made to work accurately.Exchange language
Announce message– announcement message, contains information sent by the master to all Slave devices. The slave device can use this message to select the best master (there is a BMC (Best Master Clock) algorithm for this). BMC is not that interesting. This algorithm can be easily found in the standard. The selection is based on message fields such as accuracy, variance, class, priority, etc. Let's move on to other messages.Sync/Follow Up, DelayResp, PDelayResp/PDelayFollowUp– are sent by the master; below we will look at them in more detail.
DelayReq, PDelayReq– Slave device requests.
As you can see, the Slave device is not verbose; the Master provides almost all the information itself. Sending is carried out to Multicast (if desired, you can use Unicast mode) addresses strictly defined in the standard. For PDelay messages have a separate address (01-80-C2-00-00-0E for Ethernet and 224.0.0.107 for UDP). Other messages are sent to 01-1B-19-00-00-00 or 224.0.1.129. Packets differ by fields ClockIdentity(clock ID) and SequenceId(package identifier).
Work session
Let's say the master was selected using the BMC algorithm, or the master is the only one on the network. The picture shows the procedure for communication between the main device and the synchronized one.
- It all starts with Master sending a message Sync and simultaneously records the sending time t1. There are one- and two-stage operating modes. It is very easy to distinguish them: if there is a message FollowUp– then we are dealing with a two-stage implementation, the dotted arrow shows optional messages
- FollowUp the message is sent after Sync and contains time t1. If the transfer is carried out in one stage, then Sync contains t1 in the body of the message. In any case, t1 will be received by our device. At the time of receiving the message Sync timestamp t2 is generated on Slave. Thus we get t1, t2
- Slave generates a message DelayReq simultaneously with t3 generation
- Master receives DelayReq message while generating t4
- t4 is sent to the Salve device in DelayResp message
Online messages
An exchange session like the one shown above can only be successful if the quartz generates perfectly equal frequencies for the devices being synchronized. In fact, it turns out that the clock frequency is different, i.e. On one device, in 1 second the clock value will increase by 1 second, and on another, for example, by 1.000001 second. This is where the clock divergence appears.
The standard describes an example of calculating the ratio of the time elapsed on the Master and on the Slave for a certain interval. This ratio will be the coefficient for the frequency of the Slave device. But there is an indication that adjustment can be carried out in various ways. Let's look at two of them:
- Change the clock frequency of the Slave device (example in the standard)
- Do not change the clock frequency, but for each tick of duration T the clock value will increase not by T, but by T+∆t (used in my implementation)
The second method requires calculation of ∆t. ∆t is a value that will be added to the time value every certain interval. In the figure you can see that while 22 – 15 = 7 seconds passed on the Master, 75+(87-75)/2 –(30+ (37-30)/2) = 47.5 passed on the Slave
Frequency – processor frequency, for example, 25 MHz - a processor cycle lasts 1/(25*10 6) = 40ns.
Depending on the capabilities of the device, the most suitable method is selected.
To move on to the next section, let's express offset a little differently:
PTP operating modes
Looking into the standard, you can find that there is not only one way to calculate delivery time. There are 2 operating modes of PTPv2. This E2E (End-to-End), it was discussed above, the mode is also described P2P (Peer-to-Peer). Let's figure out where to use which method and what is their difference.In principle, you can use any of the modes as desired, but they cannot be combined on the same network.
- In mode E2E delivery time is calculated from messages arriving through many devices, each of which enters the message correction field Sync or FollowUP(if two-stage transmission) the time for which the packet was delayed on this device (if the devices are connected directly, no correction is made, so we will not consider them in detail). Messages used: Sync/FollowUp, DelayReq/DelayResp
- In mode P2P In the correction field, not only the time for which the packet was delayed is entered, (t2-t1) is added to it (you can read it in the standard). Messages used Sync/FollowUp, PDelayReq/PDelayResp/PDelayRespFollowUp
End-to-End Mode
Peer-to-Peer mode
We see that some red arrows have appeared in P2P mode. These are the remaining messages that we have not addressed, namely PDelayReq, PDelayResp And PDelayFollowUp. Here is the exchange of these messages:
Delivery time error
The standard describes the implementation of the protocol in various types of networks. I was using an Ethernet network and receiving messages at the Ethernet level. In such networks, the packet delivery time is constantly changing (especially noticeable when you work with nanosecond precision). Various filters are used to filter these values.What needs to be filtered:
- Delivery time
- Bias
LP IIR (Infinite Impulse Response low-pass) filter(Infinite Impulse Response Filter) described by the formula:
, Where s– coefficient that allows you to adjust the filter cutoff.
Adjustment calculation
Let's move on to the adjustment, to the delta that will have to be added to the second value. The calculation scheme used in my system:
I used a Kalman filter to filter out the strong adjustment jitter due to network interference, I really liked it. In general, you can use any filter you like, as long as it smoothes the graph. IN PTPd, for example, filtering is simpler - the average of the current and previous values is calculated. On the graph you can see the results of the Kalman filter in my driver (the adjustment error is shown, expressed in subnanoseconds on a 25 MHz chip):
Let's move on to regulating the adjustment, the adjustment should tend to a constant, a PI controller is used. IN PTPd The clock offset is adjusted (the adjustment is based on the offset), but I use it to regulate the adjustment (a feature of the KSZ8463MLI). We see that the controller is not configured perfectly, but in my case this adjustment is sufficient:
Result of work
The result is shown in the graph. Clock offset ranges from -50ns to 50ns. Consequently, I achieved the accuracy that is mentioned in numerous articles. Of course, many small features of the implementation remained behind the scenes, but the required minimum was demonstrated.
NTP (Network Time Protocol)
This network time synchronization protocol is currently the most popular. NTP is a common method for synchronizing hardware clocks on local and wide area networks. The basic concept of the NTP protocol was published in 1988, in the so-called "version 1" RFC. The practical aspects of using this protocol on the Internet led to the appearance of “version 2” in 1989. Currently, "version 3" (Mills90) of the NTP protocol is used, based on the recommendation of RFC-1305.
The way NTP works is somewhat different from most other protocols. NTP does not synchronize all clocks connected to the network; it organizes a hierarchy of time servers and clients. Each level in this hierarchy is called stratum. Stratum-1 is the highest level. The time server at this level synchronizes itself from an external reference clock signal source: radio signals, signals from GPS/GLONASS satellite navigation systems, built-in highly stable generator, etc. The synchronization signal is then distributed across the network to several clients that are at a lower level of the stratum-2 hierarchy.
The NTP protocol allows you to compare local hardware time and adjust the clock. The synchronization accuracy using the NTP protocol is on average 10 ms. An accuracy of about 0.2 ms can often be achieved.
IRIG protocol
In 1956, the American organization Inter Range Instrumentation Group (IRIG) was tasked with standardizing time code transmission formats. Document number 104-60 defined the original IRIG protocol format. Currently, the latest version of the IRIG protocol complies with the 200-98 standard.
Description of the IRIG format
The IRIG protocol header consists of one letter followed by three numbers. Each letter or number represents an attribute of the corresponding IRIG code. The following table represents the standard IRIG protocol format types according to the 200-98 standard:
| IRIG A | IRIG B | IRIG D | IRIG E | IRIG G | IRIG H |
|---|---|---|---|---|---|
| A000 | B000 | D001 | E001 | G001 | H001 |
| A003 | B003 | D002 | E002 | G002 | H002 |
| A130 | B120 | D111 | E111 | G141 | H111 |
| A132 | B122 | D112 | E112 | G142 | H112 |
| A133 | B123 | D121 | E121 | H121 | |
| D122 | E122 | H122 |
The code formats are compiled as follows:
| first letter: Rate Designation |
A B D E G H |
1000 PPS 100 PPS 1 ppm 10 PPS 10000 PPS 1 pps |
| first digit: Form Design |
0 1 |
DC Level Shift (DCLS), width coded, no carrier Sine wave carrier, amplitude modulated |
| second digit: Carrier Resolution |
0 1 2 3 4 |
No carrier(DCLS) 100 Hz / 10 millisecond resolution 1 kHz / 1 millisecond resolution 10 kHz / 100 microsecond resolution 100 kHz / 10 microsecond resolution |
| third digit: Coded Expressions |
0 1 2 3 |
BCD, CF, SBS BCD, CF BCD BCD, SBS |
Accepted abbreviations:
BCD - Binary Coded Decimal, coding of time (HH,MM,SS,DDD)
SBS - Straight Binary Second of day (0....86400)
CF - Control Functions depending on the user application
General structure of IRIG code:
(click on the picture to enlarge)
Modulated IRIG codes
Modulated IRIG codes consist of a carrier frequency that is modulated by a time code. The carrier frequency is determined by the name of the time code format, as shown in the previous table.
Example: B123
- The second digit shows the carrier frequency (2 -> 1kHz)
- The carrier packet corresponds to the unmodulated code
- Typical duty ratio is 10:3 (can vary from 10:3 to 10:6)
The format is also widely used AFNOR NFS-87-500. It is not a type of IRIG code, but it is very similar to it.
Modulated code transmission technologies:
- 50 Ohm coaxial cable, or loaded with a high-impedance load (standard method);
- balanced twisted pair;
- analog optical transceiver (rarely used).
The IRIG code signal level is not defined in the IRIG 200-98 standard.
Unmodulated IRIG codes
- described in IRIG 200-98 standard
- DC signal level offset codes without using a carrier
Unmodulated code transmission technologies:
- TTL levels via suitably terminated coaxial cable
- differential level RS422, twisted pair
- RS232 level, shielded cable (short distances only)
- optical cable
The Prime Time company offers a wide range of precise time servers that use the NTP, IRIG and AFNOR protocols. More detailed product information
