In this work, we consider the optimization problem of finding a minimum-weight subset of vertices of a given undirected graph on n vertices whose deletion results in a d-degenerate graph. For \(d \ge 2\) , this problem is known to be constant-factor inapproximable implying that one cannot hope for anything better than bicriteria approximation algorithms. Towards this end, we give a randomized polynomial-time algorithm that for any value of the bicriteria approximation trade-off parameter \(\alpha > 1\)  and confidence parameter \(\delta \in (0, 1)\) , returns a \(2\alpha d\) -degeneracy modulator whose weight is at most \((1+\delta ) \cdot \frac{2\alpha }{\alpha -1}\) times the weight of an optimum solution with high probability. Then, we move on to the decision problem of determining if a graph G on n vertices has a d-degeneracy modulator of size at most k. For each \(d \ge 2\) , this problem is known to be W[P]-hard with respect to k and we give three FPT-approximation algorithms for solving it. These algorithms return a \(2\alpha d\) -degeneracy modulator whose size is at most k (if a k-sized d-degeneracy modulator exists) for any \(\alpha > 1\) . All our algorithms can be tuned to return a 2d-degeneracy modulator of size at most k (if a k-sized d-degeneracy modulator exists) by setting \(\alpha \) appropriately.