We study border varieties of sums of powers ( \(\underline{\textrm{VSP}}\) ’s for short), recently introduced by Buczyńska and Buczyński, parameterizing border rank decompositions of a point (e.g. of a tensor or a homogeneous polynomial) with respect to a smooth projective toric variety. Their importance stems from the role of border tensor rank in theoretical computer science, especially in the estimation of the exponent of matrix multiplication. We compare \(\underline{\textrm{VSP}}\) ’s to other well-known loci in the Hilbert scheme, parameterizing scheme-theoretic versions of decompositions. We introduce the notion of border identifiability and provide sufficient criteria for its appearance, relying on the Maclagan-Smith multigraded regularity. We link border identifiability to wildness of points. Finally, we determine \(\underline{\textrm{VSP}}\) ’s in several instances, in the contexts of tensors and homogeneous polynomials. These include concise 3-tensors of minimal border rank and in particular of border rank three, answering a question of Buczyńska and Buczyński.