Code anomalies are inevitable due to the continuously evolving nature of a software system. Code refactoring is an often-used technique for improving code comprehension and removing severe structural flaws in a software system. The problem generally is that developers are usually unaware of the importance of refactorings and often they are confused about how to apply them properly. It has been demonstrated in previous studies that code smells influence software maintainability (i.e., symptoms that indicate system degradation). However, the studies lack in the identification of code smell co-occurrences and investigating how such co-occurrences affect software quality attributes such as size, cohesion, inheritance, complexity, and coupling. In this paper, we carried out an empirical investigation to identify (i) the relation between code smell and performed refactorings along with determining how they affect quality; (ii) developer’s perception towards refactoring to identify whether the carried out refactoring efforts are mostly towards low performing (in terms of quality) classes; and (iii) co-occurrence pattern among different pairs of code smells and determining how removing co-occurrence pattern affects software quality. The empirical investigation is carried out by considering 136 releases of five state-of-the-art open-source Java software systems. Moreover, 11 categories of code smells and 71251 commits (refactoring efforts by developers) are identified and their inter-relation is investigated during empirical evaluation. We observed that refactoring efforts are not directly proportional to present code smells and different considered quality attributes show mixed behavior for carried out actual refactoring efforts by developers. Finally, removing only co-occurrence code smell patterns shows a positive impact on most of the considered software quality attributes.