Personalized or customized text-to-image (T2I) models not only produce lifelike and varied visuals but also allow users to tailor the images to fit their personal taste. These customization techniques can grasp the essence of a concept through a collection of images, or adjust a pre-trained text-to-image model with a specific image input for subject-driven. Yet, accurately capturing the distinct visual attributes of a single image poses a challenge for these methods. To address this issue, we introduce SingleDream, a novel parameter-efficient fine-tuning method which only utilizes a single reference image for attribute-driven T2I customization. A novel hypernetwork-enhanced attribute-aware fine-tuning approach is employed to achieve the precise learning of various attributes, including style, appearance and shape, from the reference image. Comparing to existing image customization methods, our method shows significant superiority in attribute-driven T2I customization generation. Code will be released.