Configuring the IEEE 802.1Qbv Time-aware shaper with deep reinforcement learning
Résumé
One of the breaking changes induced by Industry 4.0 will be the networking of production equipment. To achieve this, the Time-Sensitive Networking (TSN) set of network standards has been developed. However, this new networking paradigm will create new challenges. For example, TSN standards allow a certain level of flexibility and modularity in the data plane, however, the configuration of these standards depends on many parameters (e.g., network topology, routing strategy, critical flows requirements, etc.) making the configuration task cumbersome. The IEEE 802.1Qbv standard is among the main TSN standards that propose a mechanism allowing to achieve deterministic latency when it is appropriately configured. Today's main approach to configure this mechanism relies on exact or heuristic methods. These are adequate for closed network (when all flows are known beforehand and the network topology is fixed). However, in open networks (where flows are added to the network in an incremental way and the network topology is dynamic), the scheduling in IEEE 802.1Qbv can lead to a NP-hard problem. In this paper, we address open networks such as TSN in industrial networks with reconfigurable production lines. We propose a solution to configure the IEEE 802.1Qbv standard by using Deep Reinforcement Learning (DRL). We use simulations to train and evaluate the configuration agent.
Origine : Fichiers produits par l'(les) auteur(s)