Performing safe trajectory planning that matches perception capabilities is critical for autonomous vehicles (AV). It remains a challenge to handle uncertainty including the epistemic the aleatoric uncertainty in environmental perception in order to plan safe and accurate trajectories. We propose an integrated perception-prediction-planning algorithm for autonomous vehicles that quantifies and transfers DL-based perception uncertainties during prediction and performs prediction evaluation with uncertainty. The novelties of the approach are: 1) quantifying and transferring perceptual uncertainty to the downstream planning decision phase, which is partially extended using quantified uncertainty incorporated into a Rapidly-exploring Random Tree; 2) combining uncertainty analysis with an implicit scenario context-aware trajectory prediction framework that utilizes perceptual uncertainty as part of the implicit scenario context information; 3) integrating the proposed uncertainty-environment-aware trajectory predictor with a planning-based feasible candidate trajectory generator to capture dynamically changing perceptual states and output accurate predictions. Experimental results based on a driving behavioral dataset show that the proposed method further reduces the detour proportion of the path while ensuring safety.