Conventional Few-shot learning (FSL) techniques often struggle with overfitting and representation collapse, especially with imbalanced datasets. Existing inductive methods try to learn fine-grained features but fail to capture the correlation between the support and query embeddings. This leads to limited discriminative feature learning, resulting in sparse prototypes and overlapping decision boundaries. To address these issues, we propose Prototypical Feature Optimized Network ProFONet, a novel approach for training inductive prototypical networks that emphasizes learning inter-class and intra-class relationships while broadening decision boundaries. ProFONet integrates variance, invariance, and covariance regularization techniques to optimize feature space, resulting in more discriminative prototype clusters and robust representations. The proposed method exhibits adaptability to inductive FSL architectures and demonstrates superior performance on imbalanced datasets. By preventing collapse and reducing false positives, ProFONet outperforms state-of-the-art methods on CUB benchmark and a newly curated long-tailed medical imaging dataset GIF. The source code of ProFONet can be found at https://github.com/aj-das-research/ProFONet .