Handloom weaving in India is facing a survival crisis because of the use of power looms and the inability of many consumers to distinguish between handloom and power loom products. West Bengal tops the list of handloom weavers engaged in making sarees. This study tries to document three aspects of the struggle of handloom weavers: (a) psychological resilience, (b) the institutional framework and (c) external factors. Very few studies have been conducted on micro-entrepreneurs’ resilience to external shocks. We adopt a mixed-method approach to understand the factors that affect the survival or enterprise viability of the handloom weavers. We collect qualitative data from conducting four focus group discussions in two clusters of weavers in West Bengal–Dhaniakhali and Phulia. In the Dhaniakhali cluster, the weavers are primarily members of a cooperative society, whereas the weavers from Phulia dealt with individual traders. For the quantitative study, we surveyed 250 weavers, with 125 participants from each cluster. The qualitative study results suggest that both clusters face the onslaught of the power-loom sector and that weavers are diminishing in numbers. The quantitative study indicates that weavers with cooperative societies displayed greater resilience than the trader-dominated cluster. In addition, the cooperative-dominated cluster is distinguished from the private trader-dominated cluster by the perception of handloom quality, the ability of weavers to negotiate returns, and the effective management of business finances. Weaving businesses employing uncompensated female family labour were more resistant to shocks. Self-efficacy emerged as a significant determinant when we examined resilience regarding individual-level variables (β = 0.610). However, there was a negative correlation between the relationship with the supplier and the subjective relative efficiency of weavers (β = −0.255). The implications of the results are discussed.