The Timepix3 hybrid-pixel readout chip consists in a matrix composed of 256 × 256 square-shaped pixels with 55 µm pitch, which can be hybridized to several semi-conductors, such as Si or CdTe of thicknesses up to 5 mm. Working as an event based readout chip, it simultaneously records in each pixel the Time-over-Threshold (ToT) that gives access to deposited energy, and the time-of-arrival (ToA) with a time resolution of 1.5 ns. In this paper, we present the energy calibration of Timepix3 bump bonded with a 300 µm thick silicon sensor, and of Timepix3 bump bonded with a 1 mm thick cadmium telluride sensor. Both detectors are calibrated with a so-called per-pixel calibration using the monochromatic SOurce of Low-Energy X-rays (SOLEX from LNHB at CEA Saclay) and a set of laboratory sealed radioactive sources. Evaluation were carried out over an energy range from 6 keV to 122 keV for Timepix3 Si and from 20 keV to 1.3 MeV for Timepix3 CdTe. Results of this calibration allowed determining an energy resolution of 3.97 keV at 59.5 keV for Timepix3 Si, and 5.6 keV, 27.24 keV, and 47.4 keV at respectively 59.5 keV, 661.7 keV and 1.332 MeV for Timepix3 CdTe. It is the first time that CdTe bump-bonded Timepix3 spectral performances are evaluated for gamma-rays above MeV. The reconstruction of the total absorption peak for such high energies is possible by means of the ToA measurement which allows the identification of each interaction (scattering and absorption) of an incoming gamma-ray in the semi-conductor.