Multi-gate transistors have attracted considerable attention as a way to overcome the scaling issues of planar MOSFETs. Although the effects of structural confinement on the carrier mobilities have been discussed extensively, the transition from silicon thin films to silicon nanowires (SiNWs) has been little investigated. In this contribution, we perform quantum calculations of the size-dependent carrier mobilities in gate-all-around rectangular SiNWs with leading dimension up to 50 nm, in the non-equilibrium Green's functions (NEGF) framework. We find that when the smallest width or height falls in the sub-10 nm range, nearest neighbor corner channels tend to merge and form "side channels" with much lower mobilities. On top of the numerical results, we have derived a simple model, which bridges square NW devices with thin film devices, and describes the size dependence of the carrier mobilities in rectangular SiNWs in a wide range of dimensions.