This paper explores an analytical approach for obtaining multiple solutions for a three-wave interaction system in \((1+1)\) dimensions. We introduce a novel approach that expresses wave solutions in terms of Jacobi elliptic functions and explores specific cases involving hyperbolic functions. Additionally, this paper focuses on analyzing the linear stability of two kinds of solutions: (a) periodic and (b) one or two-hump bright solitons influenced by group velocity and group velocity dispersion. The method of separation of variables along with the ansatz method is employed to derive extract analytical solutions of this model. For linear stability analysis, the eigenvalue problem is solved using the Fourier collocation method, where Fourier coefficients are defined analytically and validated numerically. Moreover, linear stability is verified through direct numerical simulations using the pseudospectral method with special derivatives in the temporal direction (t) and the 4th-order Runge–Kutta method in the spatial direction (z), further confirmed by the Crank-Nicholson finite difference method. All these investigations within the framework of our current model yield novel insights and present breakthrough research opportunities in the realm of nonlinear optics. A key finding of this study is the discovery of stable analytical solutions, which are presented here for the first time. Furthermore, we introduce a special case known as constant magnitude wave solution and examine its modulational instability in the presence of group velocity dispersion. We also investigate the influence of group velocities and wave vector mismatches. All the results obtained are new and interesting, and the concept opens new possibilities for results in the field of nonlinear optics and nonlinear dynamics.