We propose a new integration of rule reasoners with one or more RDF stores, using selective SPARQL queries to fetch relevant data. In contrast to previous implementations that merely import results of fixed SPARQL queries, our approach relies on pure logic programs over RDF triple data. Transparent to the user, optimised SPARQL queries then are constructed and evaluated during reasoning. To ensure good performance, we develop optimisation methods that adopt ideas from logic program optimisation, including semi-naive evaluation, magic sets, and static filtering. Based on the integration of our methods into the open source rule engine Nemo, we empirically evaluate our approach with complex rule sets over large knowledge graphs.