This chapter introduces Fuzzy Logic-Driven Variable Time-Scale Prediction-based RL (FLDVTSP-RL), a data-efficient approach for multiple PiH assembly. Initially, General Value Functions (GVFs) are employed to learn predictions of variable time-scale for subsequent assembly, providing enhanced insights into the assembly environment. FLDVTSP-RL reformulates the impedance action space, calculating the baseline using impedance parameters mapped from the predicted environment using the designed FLS. The results from a dual PiH assembly experiment demonstrate the effectiveness of the learning of subsequent assembly force predictions through GVFs. Our proposed FLDVTSP-DQN and FLDVTSP-DDPG algorithms outperform typical RL algorithms without requiring parameter tuning.