We present a deep learning architecture designed to predict the success or failure of a robot control plan. The architecture accomplishes this by classifying the final state output of a future predictor using a recurrent neural network (RNN). In previous work, success or failure has been classified based on the output states of a recurrent state-space model (RSSM) and aggregated using the self-attention weights of each state. The success-or-failure predictor can serve as a constraint for enhancing robot control planning. However, it should be noted that the label of success or failure depends solely on the condition following the completion of the operation. As a result, classification should rely solely on the final output state of the future predictor. Yet, to focus on the final state in the classifier, it becomes necessary for the success-or-failure classifier to implicitly estimate the relative time from the state, since the state only represents a condition at a given time and lacks explicit time information. The challenge of time estimation can lead to poor performance. Considering this premise of success or failure, the proposed framework is designed to predict success or failure based solely on the final state of the RNN. This can be understood as a method that uses a fixed weight for aggregation over time steps. To improve performance with final state prediction, we propose the scheduling of classification loss weight. These changes are easily applicable to some reasonable RNN-based architectures. Our experiments on multi-object stacking tasks demonstrate that indeed effective for success-or-failure predictors.