Scene sketch-to-image generation demands tight alignment between abstract sketch and its generated image in terms of semantics, layout and details. However, existing diffusion-based approaches often fail to capture nuanced information conveyed by complex sketches, resulting in low-quality or unrealistic outputs. In this paper, we present a novel framework that generates high-quality scene images directly from a single abstract scene sketch, achieving robust cross-modal understanding and adaptive sketch guidance. Specifically, we convert sketch features into fine-grained textual embeddings, enabling precise semantic conditioning for diffusion model. We also incorporate layout-aware detection to extract bounding boxes and labels from sketch, fusing them with local object features via cross-attention. By combining refined textual embeddings and adaptive fusion features within denoising process, our method generates realistic scene images closely aligned with original sketches. Extensive experiments demonstrate our framework excels at interpreting abstract sketches and producing high-fidelity results with accurate semantic and spatial consistency, all from a single input sketch.