As a fundamental problem of metric-space data processing, pivot selection determines pivots, or reference points, for given metric spaces, so that each data element can be represented by its distances to pivots. Commonly, one first estimates the intrinsic dimension of data and uses it as the number of pivots, and then selects pivots according to some heuristics. Existing methods estimating intrinsic dimension are usually inaccurate or unstable. We propose a variance-based idea to use the position of the elbow in the sequence of particular variance-related statistic to estimate the intrinsic dimension. Further, we propose to incrementally select the next pivot with the maximal variance to other data after removing the covariance to other pivots by Gram–Schmidt orthogonalization. Experimental results on a comprehensive dataset show that our method estimates the intrinsic dimensions accurately and stably. Further, our pivot selection algorithm significantly outperforms typical existing algorithms in almost all the cases, by up to 70% reduction of number of distance calculations for range queries. The best existing algorithms vary for different datasets, while our algorithm outperforms the best exiting ones by about 10% on average.