Architecture of Time-Sensitive Network Configuration Management System

Architecture of Time-Sensitive Network Configuration Management System: History

View Latest Version

Please note this is an old version of this entry, which may differ significantly from the current revision.

Subjects: Others

Contributor:

Boxin Shi

Xiaodong Tu

Bin Wu

Yifei Peng

Many network applications are seeking to implement Time-Sensitive Network (TSN) technology, which not only furnishes communication transmission services that are deterministic, low-latency, highly dependable, and have ample bandwidth, but also enables unified configuration management, permitting different network types to function under a single management system. These characteristics enable it to be widely used in many fields such as industrial sensor and actuator networks, in-vehicle networks, data center networks, and edge computing. Nonetheless, TSN’s configuration management faces numerous difficulties and challenges related to network deployment, automated operation, and maintenance, as well as real-time and safety assurance, rendering it exceedingly intricate.

Time-Sensitive Network
real-time Ethernet
deterministic network
network configuration management

1. Introduction

The emergence of advanced technologies, such as automatic driving, industrial internet of things (IIoT), telemedicine, and Augmented Reality/Virtual Reality(AR/VR), has increased the demand for real-time services that require network transmission services with millisecond delays and microsecond jitters. Time-Sensitive Network (TSN), which is an Ethernet extension, offers deterministic low-latency transmission services for layer-2 networks and is therefore gaining widespread attention across various industries. For instance, the IEEE P802.1DG working group is investigating the feasibility and technical aspects of using TSN in vehicle networks, while the IEEE P802.1DP working group is focusing on applying TSN to avionics systems. In the field of industrial automation, the deployment of a large number of sensors and actuators on industrial networks is one of the challenges of TSN network configuration [1].

Currently, the standards for TSN’s data plane are mostly mature, enabling TSN devices from different manufacturers to interconnect and interoperate. However, the relevant standards for TSN’s control plane are still being enhanced. The TSN configuration management function operates on the TSN control plane and it has a significant impact on multiple aspects, including network deployment, configuration efficiency, routing and scheduling, network convergence, fault recovery, business safety of upper-layer applications, and network security. An efficient configuration management system plays a crucial role in the operational efficiency, reliability, safety, and robustness of the entire TSN network.

IEEE 802.1Qcc [2] introduces three configuration management models and provides a rough specification of network architecture. However, the protocol does not specify any specific network management behavior or protocols, making it impossible to configure and manage TSN devices produced by different manufacturers uniformly through the same system. When the network scale is large or the traffic mode is complex, TSN must provide a fast dynamic automatic configuration function to cope with the QoS requirements of different traffic types and potential network failures, which means it needs to modify the network configuration promptly during network operations without affecting the flow being transmitted in the network.

Therefore, efficient management of the entire TSN network’s behavior and resources in a unified, fast, and flexible manner is a critical research topic.

2. TSN Configuration Management Model

IEEE 802.1Qcc [2] introduces three TSN configuration management models, namely the fully distributed model, centralized network/distributed user model, and fully centralized model, as shown in Figure 1. The UNI (User/Network Interface) in the figure represents the interface between the end user and the TSN network, used to interact with user requirements and configuration related information, which is modeled in the form of U/NCI (User/Network Configuration Information) and does not rely on specific protocols.

Figure 1. Three models of TSN configuration management.

The fully distributed model does not have any entities for centralized management (Figure 1a). U/NCI hops from the Talker to the Listener along the path given by the spanning tree, and the bridges along the way perform admission control based on U/NCI and their own resource situation. UNI is located between the end user and their direct network bridge. The black solid line arrow indicates that the end user interacts with the direct bridge U/NCI through UNI. The black dashed arrow indicates the transfer of U/NCI between bridges.
The centralized network/distributed user model introduces Centralized Network Configuration (CNC) as the centralized manager of the network bridge on the basis of complete distribution (Figure 1b). The main function of CNC is to discover network topology, acquire the capabilities and resources of the bridge, calculate based on user needs, provide feedback to users, and configure the network. Similar to fully distributed, the UNI of the network centralized/user distributed model is located between the end user and its direct network bridge. End users can directly send U/NCI (black solid line arrow) to the direct bridge through UNI. The difference is that, in the network concentration/user distribution model, the direct bridge acts as an agent for CNC, so U/NCI no longer transmits hop by hop, but directly forwards between the end user’s direct bridge and CNC (black dashed arrow). CNC manages the network bridge through some remote management protocol (black dotted arrow), such as SNMP, NETCONF, and RESTCONF.
The fully centralized model further builds on the network centralized/user distributed model by introducing Centralized User Configuration (CUC) as the centralized manager for end users (Figure 1c). It can meet the demand for direct configuration management of terminal devices. CUC discovers terminals through user specific protocols (black dashed arrows), obtains the capabilities and user requirements of terminal devices, and configures them. Compared to the first two models, in the fully centralized model, UNI is located between CNC and CUC. CNC acts as the proxy for the bridge, while CUC acts as the proxy for the terminal device, and U/NCI only interacts directly between the two. CNC also manages the network bridge through some remote management protocol (black dotted arrow).

The literature on the centralized network/distributed user model is limited and it exhibits certain characteristics of the fully centralized model. Therefore, Researchers will henceforth refer to the fully centralized model and the centralized network/distributed user model collectively as the centralized model, while the fully distributed model will be referred to as the distributed model. Currently, the majority of relevant literature focuses on the centralized model, leaving little room for discussion on the distributed model in the following discourse.

3. Distributed Model

The TSN distributed model has evolved from numerous previous studies. Initially, the IEEE 802.1AVB task group developed IEEE 802.1Qat [18] based on the IEEE 802.1ak [31]. Its main content was the Stream Reservation Protocol (SRP) protocol, which was used to reserve network resources for specific audio and video streams, and avoid resource competition between audio and video streams and other streams. SRP operates in a publish–subscribe mode and supports a one-to-many mode. In addition, SRP allows modifications to existing reservation situations during runtime to meet changing requirements. IEEE 802.1Qcc [2] has improved the SRP protocol (SRPv1) to support more stream reservations.

The deep binding between the upper and lower layers of SRP protocols is inseparable, which is one of its drawbacks. The Resource Allocation Protocol (RAP) defined in IEEE P802.1Qdd is a dynamic resource reservation protocol for unicast and multicast, used to replace SRP while also providing compatibility with the original SRP protocol [21]. Although both are distributed protocols based on hop-by-hop transmission, RAP is an independent protocol that is architecturally separate from the underlying protocols, better meeting the new characteristics of TSN.

3.1. Workflow

At present, there is no relevant literature that fully describes the workflow of distributed models within research scope. Researchers have analyzed and summarized a relatively reasonable distributed model configuration management process based on the relevant description of IEEE 802.1Qcc [2] (as shown in Figure 2):

Figure 2. Process of distributed model.

Path selection: All participants in the TSN network (including end users and bridges) establish a tree logical topology through a certain protocol, such as the Spanning Tree Protocol (STP). After the tree logical topology is successfully established, the paths from the Talker to all its Listeners are determined accordingly.
Talker announce: The Talker sends declaration messages to the network using specific protocols such as SRP or RAP. The declaration message records the attribute information of the stream, including stream ID, period, rate, and frame length. This attribute information is exchanged between the databases of the bridge in a hop-by-hop propagation manner.
Listener feedback: After receiving the declaration message from the Talker, the Listener determines whether to receive the stream based on the stream attribute information. If so, it sends a feedback message requesting subscription to the stream. The feedback message returns to the Talker hop by hop along the original path, and each bridge along the way determines whether it can provide services for the flow based on its own resource situation and attaches the judgment result to the feedback message. If it is allowed to provide services for this stream, local devices will also be configured to reserve bandwidth resources for this stream.
Sending TSN stream: If the Talker receives any feedback message from the Listener indicating permission to send, it will send the TSN stream to the network and the stream will be forwarded along the original path of completing bandwidth resource reservation to the Listener subscribing to the stream.
Exit mechanism: During the TSN stream transmission process, both the Talker and Listener can choose to exit the TSN stream they are in. When a stream’s Talker or all Listeners exit, all bridges on the stream path will release the bandwidth resources reserved for them.

3.2. Research Overview

Reference [32] proposed a method based on SRPv1 to improve the fault-tolerant performance of redundant flow registration using redundancy mechanism. The research of reference [33] suggested that the current fully distributed model overestimates the delay boundary, which will lead to a decrease in the schedulability of the network. So, reference [33] proposed a flow reservation mechanism based on TDMA, which can obtain accurate delay boundaries for each flow. The authors of reference [34] provided a detailed description of the workflow of the RAP protocol in a fully distributed model in a specific network topology. The authors of reference [35] studied the dynamic configuration of TAS (IEEE 802.1Qbv [12]) based on RAP in a ring network topology.

4. Centralized Model

4.1. Basic Architecture

Researchers will introduce the advantages and disadvantages of adopting a centralized model in Section 3.4, as well as the issues that may arise from introducing a centralized controller. Overall, the introduction of centralized models in TSN configuration management remains the main research direction at present. Researchers have summarized the overall architecture of the TSN centralized model by referring to relevant research (Section 3.3.3), as shown in Figure 3. Overall, it is divided into two parts: the data plane and the control plane:

Figure 3. Architecture of centralized model.

Data Plane: The behavior of the TSN domain on the data plane is mainly regulated by many protocols such as IEEE 802.1AS [11] (Timing and Synchronization), IEEE 802.1Qbv [12] (Time-Aware Shaper), IEEE 802.1Qci [13] (Per-Stream Filtering and Policing), IEEE 802.1Qbu [14] (Frame Preemption), IEEE 802.1CB [15] (Frame Replication and Elimination), IEEE 802.1Qch [16] (Cyclic Queuing and Forwarding), and IEEE 802.1Qcr (Asynchronous Traffic Shaping) [17]. Within the scope, interested readers can refer to relevant standards. In addition, in heterogeneous multi-domain networks, there are network domains formed by non-TSN devices on the data plane. These non-TSN devices may be industrial legacy devices, ordinary IP bridges, or SDN switches that do not possess a range of capabilities specified in the TSN protocol cluster.
Control Plane: The control plane consists of various controllers. The TSN domain centralized controller is the main carrier of the TSN configuration management function, divided into CNC and CUC parts. Researchers have provided a preliminary introduction in Section 3.1. Among them, CUC includes modules such as terminal discovery, requirement collection and processing, and terminal configuration. CNC includes modules such as topology discovery, routing scheduling, admission control, resource reservation, AS management, NETCONF interface, and YANG database. Reference [36] provided a relatively comprehensive description of the functional requirements of centralized controllers, including support for multiple network management protocols, integrated SDN architecture, functional upgrades and extensions, control plane scalability, support for multiple configuration strategies, cross-domain connections, real-time dynamic automatic configuration, and handling uncertainty information, among others. In addition, support for terminal discovery, collection, and processing of end-user requirements, as well as terminal configuration and management, and support for network topology discovery are also included in the functional requirements [37]. Readers can refer to these two articles for detailed information.

In heterogeneous multi-domain networks, there are both TSN domains and non-TSN domains. On the data plane, TSN streams may need to traverse non-TSN domains. A non-TSN domain controller is used to manage devices in the corresponding non-TSN domain, which includes modules such as QoS policy, topology discovery, and flow table distribution. Different domain controllers negotiate through east–west interfaces to jointly manage the configuration of heterogeneous multi-domain networks. In addition, unified negotiation and management of various domain controllers can also be achieved through global controllers. The global controller can directly or indirectly collect and process end-user requirements. The global controller also needs to perform cross-domain clock compensation to ensure consistent clocks across different TSN domains.

4.2. Workflow

Single-domain centralized management is the most common TSN configuration management mode and is currently the main research object. Referring to Figure 3, the following is a brief introduction to the general workflow of the TSN single-domain centralized model in conjunction with Figure 4.

Figure 4. Process of centralized model. The four different colored fonts correspond to the four steps described in the text in order of serial numbers.

Topology Discovery: Centralize the controller (CNC) for network topology discovery and obtain a global view of the network by using a specific protocol such as LLDP.
User Request: The user sends an admission control request to the CUC through the user-specific protocol. The request information is directly forwarded to the CUC through a direct connection bridge, rather than being transmitted hop by hop. CUC aggregates all users’ request information and submits it to CNC via UNI.
Configure Network: After the CNC collects all the requests from all users, it calculates the routing and scheduling scheme according to the global view, and then converts the calculation results into profiles and distributes them to bridges using network management protocols such as SNMP, NETCONF, and RESTCONF. At the same time, the CNC provides feedback on the admission control status to the user and configures the user through the CUC. When the above steps are successfully completed, the Talker starts using TSN streams for data transmission.
Network Monitoring and Fault Recovery: During operation, when a network fault occurs, the neighboring nodes of the faulty device actively report the detected abnormal information to the CNC or the CNC detects that some devices are unavailable through the network topology discovery protocol. After collecting the fault information, the controller re-plans the traffic based on the global view, redistributes the configuration to the corresponding devices, and completes fault isolation or recovery.

4.3. Research Overview

Currently, the majority of research conducted on centralized models is centered on a single TSN domain scenario. This approach simplifies the model, making it more intuitive and easier to implement. However, some studies also examine complex multi-domain scenarios.

Single-domain: In the research on single-domain TSN, some literature has improved and expanded distributed protocols (RAP and SRP) to achieve faster convergence speed. Another part of the literature adopts a purely centralized approach for configuration management of single-domain TSN.

–

Improve and extend distributed protocols: Reference [34] extended the RAP protocol to make it applicable to centralized models. The specific approach is to implement a RAP-related interface in CUC for receiving and processing RAP information from terminals. Ref. [40] used an SDN controller to enhance the SRP protocol. The bridge forwards SRP messages to the SDN controller for processing and then sends them back to the bridge, which then sends them to the next hop. In this scheme, SRP messages are processed in the SDN controller instead of the bridge, but still transmitted hop by hop. Ref. [42] was simplified accordingly. After forwarding SRP messages to the SDN controller, they are directly forwarded to the destination and no longer sent back to the bridge for hop-by-hop transmission. Ref. [46] more systematically proposed the concept and architecture of Software Defined Flow Reservation (SDFR), which utilized SDN OpenFlow for SRP operations such as flow registration.

–

Pure centralized method: Ref. [38] explored the feasibility of introducing SDN in real-time Ethernet, analyzed the advantages and disadvantages of introducing SDN, and proposed a model architecture for network configuration management using SDN. Ref. [37] evaluated the feasibility of SDN OpenFlow protocol as a TSN network management protocol. Ref. [36] summarized the requirements and challenges of TSN configuration management and, based on this, proposed a flexible and scalable TSN centralized configuration management architecture combined with SDN. Ref. [39] used an SDN controller to accelerate the convergence process of clock synchronization. Ref. [41] explored the configuration management problem of wireless TSN networks based on a centralized model. Ref. [28] proposed a solution for industrial TSN configuration management using SDN and OPC UA technologies.

–

TSSDN: Reference [45] proposed the concept of TSSDN in 2016 and described its functions, including global attempts, routing, and scheduling. TSSDN is a relatively complete system solution that has been borrowed and adopted by many subsequent research institutes. The authors of [40] provided their understanding of the TSSDN architecture (as shown in Section 5.1.2). However, Ref. [48] further combined the fully centralized model described in IEEE 802.1Qcc-2018 [2] (as shown in Figure 1c), to divide the TSSDN architecture into more detailed module functions, and also introduces a unified control layer. Ref. [47] combed the challenges and future research directions of TSSDN from five aspects of reliability, performance, scalability, security, and interoperability. Ref. [44] studied the problems and challenges related to flow scheduling in TSSDN, and used integer linear programming (ILP) to solve the scheduling problem of new traffic. Ref. [43] proposed a method called $T S N μ$

The centralized configuration management architecture is similar to TSSDN and compared with TSSDN architecture.
Multi-domain: The main challenge faced by multi-domain TSN is clock synchronization and cross-domain negotiation between different TSN domains. The approach of [49,50] was similar, introducing a higher-level controller as the negotiation management module in the control plane (global controller in Figure 2), which centrally manages TSN domain controllers and indirectly manages multi-domain TSNs, enabling cross-domain clock compensation and traffic scheduling. And in the subsequent work of [49], a higher precision cross-domain clock compensation method was proposed in [52]. Cross-domain TSN flow reservation is also a challenge faced by multi-domain TSNs. Ref. [53] utilized a distributed approach by adding an east–west bound interface to the TSN domain controller. Through this interface, TSN controllers can communicate on the control plane to negotiate resource reservation for cross-domain TSN flows. However, there is no explanation in [53] on how to use east–west interfaces for cross-domain clock synchronization.
Heterogeneous network: In the research on the multi-domain problem of TSN, some have involved the problem of heterogeneous TSN. The so-called heterogeneity refers to the connection between TSN networks and non-TSN networks, and the TSN flow needs to pass through the non-TSN network. This phenomenon is common in industrial legacy networks, where some devices in the network support TSN protocol clusters while others still do not possess TSN characteristics. In addition, sometimes the TSN flow from one industrial field also needs to traverse other networks to reach the other end of the industrial field. This brings a lot of uncertainty to latency and poses new challenges in configuration management. The multi-domain TSN architecture proposed by [49,50] is also used in heterogeneous networks. And Ref. [51] utilizes the idea of distributed models to solve the configuration management problem of industrial heterogeneous networks by introducing east–west interfaces in SDN controllers.

5. Advantages and Disadvantages of Distributed and Centralized Models

Distributed models can be quickly deployed in small networks but their disadvantage is slow network convergence, making it difficult to cope with large networks and complex user needs. Currently, most relevant literature focuses on centralized models. Deterministic Network (DetNet) requires support for network management using a centralized controller [54]. Compared to distributed models, centralized models have many advantages. Reference [38] provided a relatively comprehensive description of this, covering multiple aspects such as centralized control, standardization, global view, redundancy and fault handling, multiple networks sharing a set of devices, dynamic load balancing, fault node detection and isolation, and efficient multicast.

The centralized model also has flaws. Ref. [35] believed that introducing a centralized controller in a small TSN network is not worth the loss. In a network with only one centralized controller, if the centralized controller fails, the entire network will maintain its current operating status at most, unable to add new traffic, and unable to cope with network topology changes or handle network failures. Secondly, there are many computing programs running on the centralized controller and network performance is constrained by the computing performance bottleneck of the centralized controller [55]. Especially in TSN networks, centralized controllers need to sense network topology changes in a timely manner, quickly calculate routing and scheduling plans, and configure devices. And as the network scale expands, the performance requirements for centralized controllers are also increasing. Drawing on the ideas of traditional SDN networks, introducing multiple centralized controllers to serve as hot backup and share computing tasks may become a solution, but it also needs to face the impact of hot backup switching and collaboration between multiple controllers on network operation.

This entry is adapted from the peer-reviewed paper 10.3390/jsan12040052

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.