In UO$_2$ pellets irradiated in reactor, Xe nano-bubbles nucleate, grow, coarsen and finally reach a quasi steady state size distribution (transmission electron microscope observations typically report a concentration around 10$^{-4}$ nm$^{-3}$ and a radius around 0.5nm). This phenomenon is often considered as a consequence of radiation enhanced diffusion, precipitation of gas atoms and ballistic mixing. However, 4MeV Au ion irradiation of UO$_2$ thin foils at room temperature yields a nano-void population whose size distribution reaches a similar steady state, although quasi no foreign atoms are implanted nor significant cation vacancy diffusion expected at such temperature and ion energy. Atomistic simulations performed at low temperature support the assumption of heterogeneous nucleation 25keV sub-cascades produce defect aggregates and in particular voids that grow through sub-cascade overlapping. In this work a semi-empirical model is proposed to extend these results to the simulation of the size distribution evolution of a representative defect aggregates population in a fraction of a material grain under a cascade overlap regime. To account for the damage accumulation when cascades overlap, this model is based on simple rules inferred from the atomistic simulation results. It satisfactorily reproduces the TEM observations of nano-voids size and concentration, which paves the way for the introduction of a more realistic damage term in rate theory models.