Creating new fonts is labour and time-intensive. It involves generating a glyph for each character in a given style. Few-shot Font Generation (FFG) uses generative models to generate glyphs that match the target style specified by a few reference glyphs supplied by designers. While state-of-the-art methods generate sharp glyphs, they fail to precisely capture stylistic details of various font styles, leading to inaccurate generation. We propose a refiner model that applies style refinement to the output of any base glyph generator through a series of global and local transformations using spatial transformers and deformable convolutions to better match the target font style. We propose a few-shot adaptation method to adapt the refiner to the target style using the reference glyphs only. The proposed method can be used as an add-on to any existing glyph generator to correct the local incongruencies and better match the target style. Experiments show improvement in the quality of generated glyphs and assessment scores as compared to state-of-the-art methods indicating the effectiveness of the method.