Utilizing artificial intelligence for garment design represents a key trend in the fashion design field. However, the generation of finished garments is the focus of most current garment image generation research, and the challenge concerning deeper-level hand-drawn garment design sketch generation has not been addressed. This paper proposes a generative adversarial network (DesignGAN) for generating hand-drawn garment sketches from hand-drawn garment grayscale images. Firstly, we introduce a Lpips loss and Perceptual loss functions to optimize image details and enhance image quality. Furthermore, convolutional attention modules are integrated into the generator, while multi-scale discriminators are employed to replace the original discriminator, aiming to enhance network performance and robustness. Finally, we collected and compiled a hand-drawn garment image dataset consisting of 20,020 images, including 10,010 hand-drawn garment sketches and their corresponding grayscale images, covering various styles of clothing to address the scarcity of hand-drawn garment image datasets. Experimental results demonstrate the superiority of our method in the image-to-image generation for hand-drawn garment sketches compared to several existing methods.