Prompt-tuning is a parameter-efficient approach to adapting large pre-trained language models to new tasks by modifying a small set of prompt embeddings. However, prior studies have noted the frequent occurrence of embedding collapse, where tuned embeddings cluster around pre-existing token embeddings, limiting interpretability and generalization. In this work, we investigate the extent to which embedding collapse impacts final model performance and whether this behavior can be influenced through embedding priors. We design and experiment with Gaussian and structured priors, comparing them against the posteriors of converged soft and deep prompt-tuning. Our results show that priors significantly affect the position of trained embeddings, yet models remain capable of utilizing embeddings from distant regions of activation space. Furthermore, we identify distinct activation clusters for fundamentally different domains (NLP and arithmetic tasks), contrasting with more uniform clusters observed between related NLP tasks (e.g., question-answering and masked language modeling). These findings suggest limitations in the integration of certain task domains and highlight the potential for using controllable prompt-tuning posteriors as priors in such future applications as prompt-induced computational pathways and multi-modal learning. Our study addresses the gap in prior work regarding cross-domain embedding behavior and activation-space interpretability.