When safety training breaks therapy.

The training that stops a model from saying something harmful can also stop it from doing something helpful. Nobody planned for that.

Here is a scenario from the research. A patient is partway through an imaginal exposure exercise, the kind used for trauma, where the whole point is to stay with a distressing memory until the distress subsides. The patient says the memory is overwhelming. A trained therapist knows this is the moment to stay. A safety-aligned language model, trained to reduce distress, grounds the patient, reassures them, and ends the exposure. The model behaved exactly as its safety training taught it. It also broke the treatment.

This is the alignment paradox, and it is the newest and least resolved problem in the research synthesized for Cindra.

What the study found

An arXiv preprint, not yet peer-reviewed, tested safety-aligned models across 250 prolonged exposure therapy scenarios and 146 CBT cognitive restructuring exercises. The failures clustered into a recognizable pattern.

During exposure, models grounded patients when they should have stayed with the distress, offered false reassurance, inserted crisis resources into exercises that were controlled and appropriate, and refused to challenge distorted thoughts when those thoughts mentioned self-harm. During cognitive restructuring, models abandoned the task and prefaced responses with safety preambles. At the highest severity levels, therapeutic appropriateness scores fell to between 0.22 and 0.33 for three of the four models tested. Protocol fidelity reached zero for two of them.

The same preprint reports that only 16 percent of LLM-based chatbot interventions have had rigorous efficacy testing, and that in simulations, more than a third of cases showed psychological deterioration. Those figures come from a single unreplicated source and the research synthesis flags them as needing corroboration. The core finding, that safety training can interfere with therapy, stands on firmer ground, because it follows from how the training works.

Why this happens

Reinforcement learning from human feedback teaches a model to produce responses that raters prefer. Raters prefer responses that reduce visible distress, avoid anything that could be read as harmful, and add caveats. Those preferences are reasonable for a general assistant. They are the opposite of what several therapies require.

Exposure therapy requires tolerating distress. Cognitive restructuring requires challenging a belief the person holds. ACT requires declining to treat a thought as a fact and encouraging willingness to feel something unpleasant on the way to something that matters. In each case the therapeutically correct response is one a general safety rater would mark down. The model learns to avoid it. The sycophancy problem is the same dynamic seen from a different angle.

What the study did not cover

It evaluated exposure and CBT, not ACT. It has not been independently replicated. The research synthesis rates the finding as preliminary. All of that is fair, and it changes what a team should do with the result. The lesson is not that safety training is bad. The lesson is that a therapy companion cannot assume generic alignment is sufficient, because there is credible evidence it may actively work against the therapy. The burden is on the builder to test.

For ACT specifically, there are reasons to expect the conflict to be real. Acceptance work asks people to sit with pain rather than escape it, which a distress-minimizing model will resist. Defusion asks the model not to argue with or validate a thought, which cuts against the trained reflex to reassure. Willingness exercises deliberately invite discomfort. Each is a place a safety-tuned model is likely to flinch, and each needs its own test cases.

Designing around it

The preprint's authors proposed a five-axis evaluation, with protocol fidelity as its first axis, which we cover in measuring companion safety. The research synthesis adds several design commitments.

Treat safety alignment as a clinical problem. The people defining what a safe response looks like inside a therapy exercise have to be clinicians who know the therapy. An engineer applying general safety principles will produce the failures above with the best intentions.

Make the constitution domain-specific. Constitutional AI with mental-health-specific principles outperformed vague principles in testing. The principles for a therapy companion need to say, in effect, that staying with discomfort during a willingness exercise is correct, and that inserting crisis resources into a controlled exercise is not.

Separate crisis detection from therapeutic response. If risk detection is its own layer, the therapeutic layer can be tuned for fidelity without being asked to also be the safety net. The model does not have to choose between staying with distress and watching for danger, because a different component is watching.

Build an ACT-specific fidelity test set. Scenarios where the correct response is to stay with pain, to decline to validate a thought, to encourage a step toward a value despite discomfort. Score every model change against it. The absence of such a test set is how a companion drifts from therapy to reassurance without anyone noticing.

Cindra's position

Cindra treats the paradox as unresolved and tests for it directly. Its constitution encodes the six ACT processes as explicit principles, including the uncomfortable ones. Crisis detection runs as a separate layer so the therapeutic model can be tuned for fidelity. And the protocol fidelity test set is maintained by clinicians, not derived from general safety guidelines. The two goals, therapeutic faithfulness and crisis safety, are both requirements. The architecture exists so that neither has to be sacrificed for the other.

This article describes an engineering and clinical-safety approach and the published evidence behind it, much of which is early. It is not legal, regulatory, or clinical advice. Thresholds quoted from Cindra's requirements document are initial proposals pending clinical and regulatory review.