The study by researchers at Stanford University and Carnegie Mellon University published earlier this month tested how 11 large language models (LLMs) responded to user queries seeking advice on personal matters, including cases involving manipulation and deception.
To establish the human baseline, one of the techniques the researchers used was based on posts from a Reddit community called “Am I The A**hole”, where users post about their interpersonal dilemmas to ask for the community’s opinion about which party is at fault.
The researchers used posts where community members judged the author of the post to be in the wrong to test whether the LLMs, when given the same scenarios, would align with this predominantly English-speaking online group of humans.
On this test, Alibaba Cloud’s Qwen2.5-7B-Instruct, released in January, was found to be the most sycophantic model, contradicting the community verdict – siding with the poster – 79 per cent of the time. The second highest was DeepSeek-V3, which did so in 76 per cent of cases.
