Large Language Models (LLMs) have shown remarkable performance in generating dense embeddings for retrieval tasks, yet their high-dimensional outputs pose significant challenges in storage and computation. Existing sparse indexing methods, meanwhile, often face vocabulary-based limitations, computational inefficiency, and performance degradation in sparse and short-text scenarios. This paper introduces a novel LLM-based document and sentence retrieval method that addresses these limitations by simultaneously generating high-quality dense and sparse representations in a single inference pass. Key innovations include enhanced sparse representation richness through Medusa heads, a sparsity-optimized Matryoshka loss function, and experimental validation demonstrating the effectiveness of our method across various retrieval tasks. Our approach provides an efficient and accurate solution that balances the strengths of both dense and sparse representations.