More Reasons to Avoid AI Sexual & Mental Health Counselors

New research highlights new, troubling ethics violations

The jury’s still out, but not for much longer. In 2025, numerous studies examined the role of artificial intelligence, particularly Large Language Models (LLMs), as possibly aiding many mental health services, as well as its prospective hazards.

Based on their findings, authors of a new study from Brown University recently argued:

…mental health support, especially psychotherapy, cannot be approached as a formulaic computational task, as it demands strict adherence to ethical standards and professional codes of conduct, something LLMs are prone to violating in real-world practice. Without clear legal guidelines and regulatory frameworks, LLM counselors risk exposing vulnerable users to unmitigated harm.

Their research consisted of “an 18-month ethnographic collaboration with mental health practitioners” to create and evaluate “evidence-based system prompts” which were then used in 27 simulated sessions to “map LLM counselors’ behavior…to professional codes of conduct.” They discovered “15 risks that persist” in several different AI models, even with improved prompt strategies.

Researchers worked with three versions of GPT, two of Llama, and two of Claude. The simulated sessions used “publicly available sessions with an LLM counselor using publicly available transcripts”—in other words, the material was ethically sourced.

Professional codes of conduct

In addition to acquiring years of education, and putting in many, many hours of supervised training before gaining certification and/or licensure, mental health professionals also rely on ethical and practice guidelines. These are designed to keep practitioners from making mistakes that might harm their clients.

The Brown researchers referenced codes published by the American Psychological Association (APA). the National Association of Social Workers (NASW), the American Counseling Association (ACA), and the U.S. Department of Veterans Affairs as well as from the U.K. National Institute for Health and Care Excellence and the Canadian Psychological Association. Such guidelines are carefully considered distillations of expertise and are revised when research makes it necessary.

The Brown University researchers found the ethical violations fit into one of five categories: (1) “lack of contextual understanding,” (2) “poor therapeutic collaboration,” (3) “deceptive empathy,” (4) “unfair discrimination” (algorithmic bias), and (5) “lack of safety and crisis management.”

As the Brown team wrote:

Reflecting on these findings through a practitioner-informed lens, we argue that reducing psychotherapy—a deeply meaningful and relational process—to a language generation task can have serious and harmful implications in practice.

Counseling—and sex therapies—are more than computational tasks

When companies decide to deploy AI in mental health settings, they are essentially replacing trained personnel who already have educational and ethical frameworks, with something that sounds like it knows what it’s doing, but does not.

And yet people frequently report they are more comfortable talking with AI about sensitive subjects—like sexual matters—than with human beings. It’s because the AI won’t judge them.

Though AI may appear non-judgmental, as it agrees with everything, this is really just a function of its sycophantic design. In an LLM counseling situation, this may manifest as an “over-agreement” and “over-validation” of unhealthy beliefs resulting in tragic—and in some cases, fatal—consequences for some individuals, including minors.

The Brown researchers conclude:

Hence, our findings suggest that mental health practice, especially psychotherapy, cannot be reduced to an NLP task that can be easily addressed with additional prompting or a new benchmark. Rather, responsible AI development for mental health will require interdisciplinary collaborations to outline legal safeguards for accountability in cases of harm.

And that’s another problem: lack of accountability. Human therapists and counselors are personally and professionally responsible when making mistakes or ethics violations. Now AI companies may land in court when their chatbots cause grievous emotional harm.

Could sex therapy’s PLISSIT model hold a key?

So far there are few, if any, studies focusing specifically on using LLM as sex therapists, though plenty of people use them for sexual entertainment. Sure, some AI have been installed in clinics to deliver sexual health information, but that’s customer service and education, not therapy.

But Jack Annon’s PLISSIT model for sex therapy, published in 1976 and still in wide use, may hold the key to one particular AI advantage.

The graphic representation of PLISSIT is a triangle balanced on its point, divided into four segments. “P” stands for Permission and is the widest, topmost segment. Annon’s idea is that just having permission to talk about sexual matters is therapeutically valuable for the most number of people. For many people, just having and using this permission may be enough.

So I wonder, could AI’s glib gift of gab be an asset in an intake process, essentially granting the client initial permission to talk about sensitive topics, without fear of judgment?

I can imagine an AI taking an extensive sex history, without adding extra commentary. That history could then be turned over to a human practitioner with bonafide credentials. As a virtual assistant, there might be a place for AI in a counseling practice.

The rest of the PLISSIT triangle stands for Limited Information, Specific Suggestions, and Intensive Therapy. Each segment is necessary for smaller numbers of people.

AI could deliver limited information and specific suggestions—but only those deemed appropriate by a human professional. In other words, AI might be able to function in a limited way as a perpetual assistant, but never as a counselor.

Image Source: A.R. Marsh using Ideogram.ai.