The latest advancements in Augmented Reality (AR) and Diminished Reality (DR) have allowed the development of many consumer-oriented applications (such as sales and driving aid, or education). To increase the realism in rendering, estimating the illumination in the scene is a key element. A lot of works tackle this problem but rarely discuss the color of the reconstructed illumination. The Dichromatic Model indicates that the specular component is not affected in color by the texture underneath and holds the light source's color. Though theoretically sound, in practice consumer cameras are subject to nonlinear behaviors which change RGB ratios and create inconsistencies when estimating the illumination. In this paper, we study the conditioning and limits of inverting local illumination models while relying on the Dichromatic Model. We show that the reconstructed specular component has an inconsistent color because it changes depending on the surface's colors.