Offline reinforcement learning enables agents to learn from fixed datasets without requiring online interaction, offering clear advantages in scenarios where collecting real-world data is costly or risky. However, policies trained solely offline often lack robustness to action-space perturbations due to the absence of exploration during training. This study explores whether online fine-tuning with randomly perturbed actions can enhance robustness. We consider two types of models: an actor-critic method and a Transformer-based model. We evaluate the models in simulated the legged robot environment, where perturbations are applied to joint torque outputs to emulate control faults. Fine-tuning significantly improves robustness in the actor-critic method, whereas only marginal gains are observed in the Transformer-based model, which primarily relies on trajectory imitation. These results reveal the difficulty of adapting Transformer-based models to perturbed environments. Furthermore, they suggest that integrating actor-critic mechanisms into such models may provide a promising direction for improving robustness within the offline-to-online RL framework.