Many Artificial Intelligence (AI) tools have been developed for suicide prevention. These technologies are wide ranging, from Language Models (LM) in chatbots for therapy or triaging, computer vision algorithms designed to detect specific human motions to the use of limited emotion theories to detect emotional states. Both their design and implementation raise ethical concerns, especially considering that - unlike clinical research on human subjects - they are not necessarily subjected to any regulations or ethics governance structures, nor does the design or implementation require any formal expertise in mental health. Considering the nature of such technologies, the high-risk area they are applied in, and the explicit targeting of vulnerable populations, this is concerning. In this work, we aim to (a) give an overview of such tools, (b) provide some insight into key ethical concerns, (c) demonstrate that these technologies do not necessarily comply with existing guidelines regarding human subject research, although some of the tools would most likely be subjected to it, had they been done in a clinical setting (d) argue that there is an unmet need for specific ethics guidelines for research, scientists, and practitioners working at the intersection of AI and suicide prevention, and (e) provide such guidelines.