The WEST (W Environment in Steady-state Tokamak) divertor serves as the primary element for heat exhaust and contributes critically to plasma control. The divertor receives intense heat fluxes, potentially leading to damage to the plasma facing units. Hence, it is of major interest for the safety of divertor operation to detect and characterize the hot spots appearing on the divertor surface. This is done through the use of infrared (IR) cameras, which provide a thermal mapping of the divertor surface. In this work, a knowledge-informed divertor hot spot detector is demonstrated, that explicitly accounts for hot spot structure and temperature repartition. A novel neural network, termed as Constrained U-Net, is proposed, which uses as input the bounding boxes of hot spots from prior automatic detection. The Constrained U-Net addresses jointly image segmentation and regression of physical parameters, while remaining compatible with the practical constraints of real-time use. The detector is trained on simulated data and applied to real-world infrared images. On simulated images, it yields a precision of 0.98, outperforming a classical U-Net, and Max-Tree. Visual results obtained on real-world acquisitions from the WEST Tokamak illustrate the reliability of the proposed method for safety studies on hot spots.