  Accurate camera pose estimation is a fundamental requirement for numerous
applications, such as autonomous driving, mobile robotics, and augmented
reality. In this work, we address the problem of estimating the global 6 DoF
camera pose from a single RGB image in a given environment. Previous works
consider every part of the image valuable for localization. However, many image
regions such as the sky, occlusions, and repetitive non-distinguishable
patterns cannot be utilized for localization. In addition to adding unnecessary
computation efforts, extracting and matching features from such regions produce
many wrong matches which in turn degrades the localization accuracy and
efficiency. Our work addresses this particular issue and shows by exploiting an
interesting concept of sparse 3D models that we can exploit discriminatory
environment parts and avoid useless image regions for the sake of a single
image localization. Interestingly, through avoiding selecting keypoints from
non-reliable image regions such as trees, bushes, cars, pedestrians, and
occlusions, our work acts naturally as an outlier filter. This makes our system
highly efficient in that minimal set of correspondences is needed and highly
accurate as the number of outliers is low. Our work exceeds state-ofthe-art
methods on outdoor Cambridge Landmarks dataset. With only relying on single
image at inference, it outweighs in terms of accuracy methods that exploit pose
priors and/or reference 3D models while being much faster. By choosing as
little as 100 correspondences, it surpasses similar methods that localize from
thousands of correspondences, while being more efficient. In particular, it
achieves, compared to these methods, an improvement of localization by 33% on
OldHospital scene. Furthermore, It outstands direct pose regressors even those
that learn from sequence of images
