Non rigid surface registration consists in estimating the deformation of a known surface between two images, usually by fitting a warp such as a Thin-Plate Spline or a Free-Form Deformation. Common techniques are split in two categories: feature-based surface detection i.e. estimation of a potentially important deformation from an image and a flat source template, and pixel-based surface tracking where important deformations can be estimated over a video sequence as long as the frame to frame steps are small. Our contribution consists in bridging the two worlds by introducing a new data term robustly merging feature and pixel-based costs in a pyramidal variational approach. By using a robust estimator we achieve an implicit optimal filtering of features and automatic balancing between the two terms. Our goal is to directly estimate a deformation between an image I and a given flat template I0. This deformation is parametrized by the displacements u of the control points of a Free-Form Deformation warp [4]. The image in I of the point q from I0 is W(q,u). For the sake of clarity and to allow easier comparison with feature-based methods, we adopt a feature-filtering model for our cost function