Large language models (LLMs) often encode subtle biases that reflect historical disparities in their training data. While many studies evaluate bias solely based on model outputs, the internal mechanisms that give rise to such biases remain underexplored. In this work, the framework Bias-CAVs is proposed, an innovative framework that extends the concept activation vector methodology to probe and explain internal LLM representations for bias. This approach conducts a layer-wise analysis to quantify bias projections, revealing where bias is introduced, amplified, or mitigated within the network. In a two-stage process, activations are first extracted from key layers of diverse LLM architectures (e.g., LLaMA-3, Mistral, Phi, and Gemma) and then a linear classifier (via logistic regression) is trained on standardized activations to create Bias-CAVs that distinguish between biased and neutral representations. The findings across gender, race, profession, and political domains contribute to both the understanding of LLM internal bias propagation and the development of more explainable debiasing interventions.