We report on the correlations between magnetic anisotropy and strain state in CoFe$_2$O$_4$ ultrathin films grown on MgAl$_2$O$_4$(100) and MgAl$_2$O$_4$(111) substrates. By local strain analysis using the geometric phase method, a significant in-plane compressive strain is observed for the (001) orientation while a full relaxation is detected for the (111) orientation. The relaxation process in CoFe$_2$O$_4$(111) layers induces interface dislocations and a large amount of antiphase boundaries while a pseudomorphic growth is observed for the (001) direction, decreasing significantly the density of antiphase boundaries. By comparing the magnetoelastic energy terms, the correlation between strain state and resultant magnetization is discussed.