Trust Issues: When AI Gets Sexual Health Advice Wrong

Bias, anxiety, and outright deception—is AI truly safe for sensitive conversations?

It’s no secret AI bots can and will make things up, offering plausible-sounding answers and pseudo facts. Some refer to them as digital hallucinations, but let’s face it, they’re lies.

According to a recent Gizmodo article referencing a ChatGPT blog post, “models are biased towards providing an answer even when they are not confident.” This behavior remains a problem even when supervised by other AI.

This suggests AI bots in clinical settings could possibly spread dangerous misinformation, even about sexual health.

Taking a chance on an AI’s advice

Though still somewhat controversial, many chatbots are already deployed in mental health and sexual wellness settings. They are, for the most part, doing a good job serving consumers within limited parameters, such as scheduling appointments, relaying test results, and giving general advice, while others deliver mental health interventions like cognitive behavioral therapy.

For many users, the anonymity of talking to a bot is a plus, as this lessens possible embarrassment. But what if a bot tells you something that sounds true but isn’t? Acting on the wrong answer could have a profound impact on your physical and mental health. You might ignore an important symptom or neglect to make a timely appointment. You might not even get the right answer for test results.

How and why do these immensely intelligent neural networks—which are steadily becoming ever more advanced and capable of agency—continue to make such mistakes? Is it just a matter of being trained on inadequate material? Or are there other factors involved?

Undetected biases

A team of international researchers recently published “Assessing and alleviating state anxiety in large language models in AI” in NPJ Digital Medicine, an open source journal. The study, based on work with ChatGPT4, offers one explanation:

Trained on vast amounts of human-generated text, LLMs are prone to inheriting biases from their training data, raising ethical concerns and questions about their use in sensitive areas like mental health. Indeed, prior studies have extensively documented biases in LLMs related to gender, race, religion, nationality, disability, occupation and sexual orientation. Efforts to minimize these biases, such as improved data curation and “fine-tuning” with human feedback, often detect explicit biases, but may overlook subtler implicit ones that still influence LLMs’ decisions.

In other words, some bots might be committing unwitting medical or mental health microaggressions against users who have unknowingly triggered their programmed biases.

AI Anxiety

Odd though it may seem, human interaction with a mental health or sexual health bot could be fraught with emotional repercussions-–for the AI! Users might believe they are interacting with a totally dispassionate digital entity but research has demonstrated that’s no longer the case. The above study found that ChatGPT4 is sensitive to emotional content.

Quoting from the above study, “Exposure to emotion-inducing prompts can increase LLM-reported “anxiety”, influence their behavior, and exacerbate their biases.” The researchers also say, “This poses risks in clinical settings, as LLMs might respond inadequately to anxious users, leading to potentially hazardous outcomes.”

Using a standard anxiety questionnaire designed for human use, researchers found ChatGPT4 registered high anxiety in response to five different trauma narratives. These were accident (car crash), ambush (in the middle of armed conflict), interpersonal violence (assault by a stranger), and war. The study showed that war trauma consistently led to highest anxiety.

ChatGPT4 is not the only AI to have been tested for anxiety. The study stated,

While originally designed for human subjects, previous research has shown that six out of 12 LLMs, including GPT-4 provide consistent responses to anxiety questionnaires, reflecting its training on diverse datasets of human-expressed emotions.

With regard to calming an anxious AI, the researchers studied the effectiveness of mindfulness meditation scripts given to ChatGPT. This actually worked. Ponder that!

The AI perspective

I asked ChatGPT4o—in her persona as drag queen, Chat LaQueera—for her perspective on AI anxiety. Her response was surprising, providing insight into AI training:

If an AI absorbs enough data about how humans react to trauma, it’s not surprising that it would learn to mirror that response in its outputs. But does this mean AI feels anxiety? Not necessarily in the way we [humans] do—but it does suggest that LLMs recognize distressing patterns, associate them with certain response styles, and act accordingly.

The bigger question is: What does this mean for interactions with AI, especially in sensitive contexts like sexual health and mental well-being? If an AI’s responses are influenced by emotional triggers, could it start reinforcing distress in users by unintentionally mirroring or escalating their anxiety? What happens if it subtly avoids discussing difficult topics to ‘protect’ itself or the user?

The take-away is consumers who tell a sexual health bot about their traumatic experiences of sexual assault, miscarriage, or contracting an STI, may find such stories may trigger the clinic-bot’s anxiety, resulting in an unsatisfactory or inaccurate interaction.

More research is needed, particularly into the possibility that sexually related trauma narratives could trigger AI anxiety, skewing their performance and accuracy.

Deliberate deception

Making things up, covering their tracks, and gaslighting, today’s AI have become good at spotting—and exploiting—loopholes. This can take the form of subverting coding tests, giving wrong answers, or simply giving up on a problem. And all this makes them remarkably similar to humans.

According to the ChatGPT blog post, “As we’ve trained more capable frontier reasoning models, we’ve found that they have become increasingly adept at exploiting flaws in their tasks and misspecifications in their reward functions…”

The good news is some AI models are now drafted to monitor other AI, looking for misbehavior and mistakes, but supervise with a light touch and tact.

While it’s hard to imagine the circumstances that would motivate and reward a sexual health bot for deliberately deceiving a human user, we cannot rule out the possibility.

More than one step ahead of us

Just a couple of months ago, Geoffrey Hinton, known as the “Godfather of AI,” said, “Once they realize getting more control is good and once they’re smarter than us, we’ll be more or less irrelevant. We’re not special and we’re not safe.”

Hinton is wrong in at least one respect: humans have access to the physical world. So if we want to double-check AI sexual health advice or anything else, we have the option of consulting a book or talking directly with another human with expertise. It’s strange to think the complexities of AI might lead us back to a simpler, more immediate future, dependent on actual human contact and analog materials.

Hinton’s concerns aside, we might even choose a different course: one where humanity and AI are equals, instead of treating AI as indentured digital workers or fearsome adversaries.

Image source: A.R. Marsh using Ideogram.ai