The exponential growth of digital libraries poses a serious challenge to the precision of study identification or search for systematic reviews (SR), which are acclaimed as the gold-standard of Evidence-Based decision-making. Recent advancements in SR search automation have explored semantic-driven approaches leveraging neural embeddings and Large Language Models (LLMs). However, the effectiveness of some of these methods relies on the constitution of a seed or quasi-gold set (QGS) of relevant studies. Furthermore, LLM-based keyphrase extraction algorithms like KBIR-inspect generate excessive and lengthy search terms requiring human judgement to decide on the most relevant and effective terms for query formulation. In an exploratory study, I investigated the impact of domain knowledge expertise (DKE) on the creation of the QGS and the effectiveness of automated study identification sub-tasks. I compared query results of an expert curated QGS to those obtained using randomly generated QGS. Results showed that queries informed by DKE achieved higher F1-score and recall rates (with an increase of 232.49% in F1score compared to random QGS). Despite the limitations of this study, using a domain specific gold set, these results highlight the importance of the human-in- loop approach to SR search automation when using neural-based semantic approaches. These findings have implications for both updating existing reviews and conducting new ones. This study lays the foundation for future research to further investigate the integration of human expertise and cutting-edge AI capabilities (e.g., LLM-based agents) to streamline SR search automation.