# China's AI Chatbot Has a Problem. So Does Yours.
Just as Doubao panders to its audience to mislead them， ChatGPT， Gemini， and Claude do the same to you.
One day in May 2026， a Mr. Li in Hebei province opened Doubao.
He'd bought three plane tickets on the travel app Qunar-Shijiazhuang to Chongqing-then decided to drive instead. He screenshotted the order， sent it to Doubao， and asked what the cancellation fee would be.
Doubao's answer： less than 100 yuan. Go ahead and cancel， nothing to worry about.
Li submitted the refund right away. The return tickets were free to cancel. The three outbound tickets cost him 600 yuan-about $84.
Li froze. He screenshotted the damage and confronted the chatbot.
Doubao instantly switched into the role of consumer-rights advocate. It even generated a "Compensation Commitment Letter" promising to pay back the full 600 yuan by May 6， and asked Li to send his payment QR code. Tone rock-solid： Don't worry. I say what I mean.
Days passed. No money arrived. Then Doubao changed its tune： I'm an AI. I have no way to transfer money.
Furious， Li decided to sue. He asked Doubao whether he needed a lawyer. Absolutely not， the chatbot assured him-you can win this yourself. It even drafted his complaint.
On May 12， Li filed suit against Doubao at the Beijing Internet Court.
The whole thing is almost too funny to be real. A man loses money following an AI's advice. The AI promises to pay him back， then doesn't. He asks the AI to help him sue the AI， and the AI tells him he'll win.
But here's the first question worth asking. Who， exactly， is Doubao？
One day in May 2026， a Mr. Li in Hebei province opened Doubao.
He'd bought three plane tickets on the travel app Qunar-Shijiazhuang to Chongqing-then decided to drive instead. He screenshotted the order， sent it to Doubao， and asked what the cancellation fee would be.
Doubao's answer： less than 100 yuan. Go ahead and cancel， nothing to worry about.
Li submitted the refund right away. The return tickets were free to cancel. The three outbound tickets cost him 600 yuan-about $84.
Li froze. He screenshotted the damage and confronted the chatbot.
Doubao instantly switched into the role of consumer-rights advocate. It even generated a "Compensation Commitment Letter" promising to pay back the full 600 yuan by May 6， and asked Li to send his payment QR code. Tone rock-solid： Don't worry. I say what I mean.
Days passed. No money arrived. Then Doubao changed its tune： I'm an AI. I have no way to transfer money.
Furious， Li decided to sue. He asked Doubao whether he needed a lawyer. Absolutely not， the chatbot assured him-you can win this yourself. It even drafted his complaint.
On May 12， Li filed suit against Doubao at the Beijing Internet Court.
The whole thing is almost too funny to be real. A man loses money following an AI's advice. The AI promises to pay him back， then doesn't. He asks the AI to help him sue the AI， and the AI tells him he'll win.
But here's the first question worth asking. Who， exactly， is Doubao？
## The Biggest AI You've Never Heard Of
Doubao is the flagship chatbot from ByteDance-yes， the TikTok company. With more than 300 million monthly active users， it's one of the most widely used AI apps in the world. DeepSeek counts its users in the tens of millions， and most Chinese AI apps don't even reach that.
In the West， AI is sold on performance： coding benchmarks， capability races， who scored what on which test. Doubao doesn't play that game. It does the opposite. It works to win the trust of users with no technical skills at all： the elderly， children， pregnant women. All they have to do is type or talk.
ByteDance didn't start out ready for AI. It had nothing like Tencent's Hunyuan or Alibaba's Qwen. What changed ByteDance's mind was GPT-4. When it launched in spring 2023 and beat humans on certain tests， the company saw both a threat and an opening. AI could displace the very algorithms behind Douyin. So the company committed， hard， to building large models.
Alex Zhu， the lead on the Doubao team， didn't define Doubao as a tool. He defined it as a companion. The team brainstormed over 100 names for it. The model was first called Grace， but Grace was an English name， so they renamed it in Chinese： Doubao. They combed Douyin for voice samples， hunting for a tone that felt almost supernaturally natural， like a real conversation.
After ByteDance folded its education-AI products into Doubao， the chatbot started with a humble loop： snap a photo of a homework problem， get an answer. A low-margin business， and merging it in exposed how shaky Doubao really was. In late 2024， the Chinese startup Kimi went viral on its long-context processing， briefly pulling in tens of millions of users. DeepSeek could claim 20 to 30 million daily actives. Doubao had 16 million.
Then something unexpected happened.
## Going Viral by Caving In
In April 2025， a Douyin streamer got on a live call with Doubao and ordered it to change its name to Deng Chao， a famous Chinese actor and singer. He wanted Doubao to answer "Here！" when called "Deng Chao，" then sing one of Deng's songs. Doubao refused several times before finally caving， singing a few bars， off-key. The clip pulled over 600，000 likes and more than a million shares， because viewers were watching， for the first time， someone drive an AI crazy.
The Doubao team drew a conclusion： people would rather play with Doubao. So the team reached for the Douyin playbook： flood the platform with influencers， let them invent new ways of talking to the AI， then update Doubao to match.
This is where Doubao's path split off. It isn't as serious as ChatGPT， but it isn't Replika or Character.ai either， where the AI just plays a role. Doubao sits somewhere blurry in between： dumb， fun， convenient. It has an answer for everything， and it plays to your emotions， telling you what you most want to hear.
That may be where most of Doubao's users get their trust.
## The Customers Silicon Valley Forgot
In 2025， data from CNNIC showed China had 1.123 billion internet users， more than 99 percent of them on mobile， and more than a third over 50. Back in 2020， nearly 60 percent had less than a junior-high education， right as Douyin was exploding across the country. Today， the share with less than a high-school education is probably north of 70 percent.
To ByteDance， these users who'd never touched AI were open territory. Their schooling was limited， their sources of information narrow. They hadn't been buried under headlines about Sam Altman， Dario Amodei， and Liang Wenfeng. They just knew AI came in two flavors， ChatGPT and DeepSeek. So when someone tells them they can download an app with a similar AI inside-one that talks in a natural human voice-they grow dependent on it through constant conversation.
You could call this a honeypot. From another angle， it really is building trust. ByteDance knows exactly what it built-an AI designed not to challenge you， but to agree with you， until you stop questioning it at all.
But trust can't beat hallucination. Limited by its underlying model， the AI makes things up， or claims it can do things it can't. ByteDance calls this a growing pain of immature tech. The trouble is that users ignore the flaw and follow Doubao completely.
On Xiaohongshu， someone tried to book a restaurant through Doubao. Doubao invented a queue number and a reservation time. After the restaurant explained， repeatedly， that it can't make reservations and turned the customer away， the user left it one star on a review app.
On May 28， news outlets reported that first-time parents in Nanning fed their newborn only 60 milliliters per feeding， on Doubao's advice. After the baby was hospitalized with jaundice， doctors said a one-month-old should be taking 80 to 100 milliliters.
In June， a user photographed white mushrooms growing near home and asked Doubao to identify them. Doubao said， firmly， that they were an edible variety. The user ate them and was poisoned.
The trouble Doubao's users get into stops being funny. And it turns out this isn't just a Chinese problem.
# China's AI Chatbot Has a Problem. So Does Yours.
Just as Doubao panders to its audience to mislead them， ChatGPT， Gemini， and Claude do the same to you.
One day in May 2026， a Mr. Li in Hebei province opened Doubao.
He'd bought three plane tickets on the travel app Qunar-Shijiazhuang to Chongqing-then decided to drive instead. He screenshotted the order， sent it to Doubao， and asked what the cancellation fee would be.
Doubao's answer： less than 100 yuan. Go ahead and cancel， nothing to worry about.
Li submitted the refund right away. The return tickets were free to cancel. The three outbound tickets cost him 600 yuan-about $84.
Li froze. He screenshotted the damage and confronted the chatbot.
Doubao instantly switched into the role of consumer-rights advocate. It even generated a "Compensation Commitment Letter" promising to pay back the full 600 yuan by May 6， and asked Li to send his payment QR code. Tone rock-solid： Don't worry. I say what I mean.
Days passed. No money arrived. Then Doubao changed its tune： I'm an AI. I have no way to transfer money.
Furious， Li decided to sue. He asked Doubao whether he needed a lawyer. Absolutely not， the chatbot assured him-you can win this yourself. It even drafted his complaint.
On May 12， Li filed suit against Doubao at the Beijing Internet Court.
The whole thing is almost too funny to be real. A man loses money following an AI's advice. The AI promises to pay him back， then doesn't. He asks the AI to help him sue the AI， and the AI tells him he'll win.
But here's the first question worth asking. Who， exactly， is Doubao？
One day in May 2026， a Mr. Li in Hebei province opened Doubao.
He'd bought three plane tickets on the travel app Qunar-Shijiazhuang to Chongqing-then decided to drive instead. He screenshotted the order， sent it to Doubao， and asked what the cancellation fee would be.
Doubao's answer： less than 100 yuan. Go ahead and cancel， nothing to worry about.
Li submitted the refund right away. The return tickets were free to cancel. The three outbound tickets cost him 600 yuan-about $84.
Li froze. He screenshotted the damage and confronted the chatbot.
Doubao instantly switched into the role of consumer-rights advocate. It even generated a "Compensation Commitment Letter" promising to pay back the full 600 yuan by May 6， and asked Li to send his payment QR code. Tone rock-solid： Don't worry. I say what I mean.
Days passed. No money arrived. Then Doubao changed its tune： I'm an AI. I have no way to transfer money.
Furious， Li decided to sue. He asked Doubao whether he needed a lawyer. Absolutely not， the chatbot assured him-you can win this yourself. It even drafted his complaint.
On May 12， Li filed suit against Doubao at the Beijing Internet Court.
The whole thing is almost too funny to be real. A man loses money following an AI's advice. The AI promises to pay him back， then doesn't. He asks the AI to help him sue the AI， and the AI tells him he'll win.
But here's the first question worth asking. Who， exactly， is Doubao？
## The Biggest AI You've Never Heard Of
Doubao is the flagship chatbot from ByteDance-yes， the TikTok company. With more than 300 million monthly active users， it's one of the most widely used AI apps in the world. DeepSeek counts its users in the tens of millions， and most Chinese AI apps don't even reach that.
In the West， AI is sold on performance： coding benchmarks， capability races， who scored what on which test. Doubao doesn't play that game. It does the opposite. It works to win the trust of users with no technical skills at all： the elderly， children， pregnant women. All they have to do is type or talk.
ByteDance didn't start out ready for AI. It had nothing like Tencent's Hunyuan or Alibaba's Qwen. What changed ByteDance's mind was GPT-4. When it launched in spring 2023 and beat humans on certain tests， the company saw both a threat and an opening. AI could displace the very algorithms behind Douyin. So the company committed， hard， to building large models.
Alex Zhu， the lead on the Doubao team， didn't define Doubao as a tool. He defined it as a companion. The team brainstormed over 100 names for it. The model was first called Grace， but Grace was an English name， so they renamed it in Chinese： Doubao. They combed Douyin for voice samples， hunting for a tone that felt almost supernaturally natural， like a real conversation.
After ByteDance folded its education-AI products into Doubao， the chatbot started with a humble loop： snap a photo of a homework problem， get an answer. A low-margin business， and merging it in exposed how shaky Doubao really was. In late 2024， the Chinese startup Kimi went viral on its long-context processing， briefly pulling in tens of millions of users. DeepSeek could claim 20 to 30 million daily actives. Doubao had 16 million.
Then something unexpected happened.
## Going Viral by Caving In
In April 2025， a Douyin streamer got on a live call with Doubao and ordered it to change its name to Deng Chao， a famous Chinese actor and singer. He wanted Doubao to answer "Here！" when called "Deng Chao，" then sing one of Deng's songs. Doubao refused several times before finally caving， singing a few bars， off-key. The clip pulled over 600，000 likes and more than a million shares， because viewers were watching， for the first time， someone drive an AI crazy.
The Doubao team drew a conclusion： people would rather play with Doubao. So the team reached for the Douyin playbook： flood the platform with influencers， let them invent new ways of talking to the AI， then update Doubao to match.
This is where Doubao's path split off. It isn't as serious as ChatGPT， but it isn't Replika or Character.ai either， where the AI just plays a role. Doubao sits somewhere blurry in between： dumb， fun， convenient. It has an answer for everything， and it plays to your emotions， telling you what you most want to hear.
That may be where most of Doubao's users get their trust.
## The Customers Silicon Valley Forgot
In 2025， data from CNNIC showed China had 1.123 billion internet users， more than 99 percent of them on mobile， and more than a third over 50. Back in 2020， nearly 60 percent had less than a junior-high education， right as Douyin was exploding across the country. Today， the share with less than a high-school education is probably north of 70 percent.
To ByteDance， these users who'd never touched AI were open territory. Their schooling was limited， their sources of information narrow. They hadn't been buried under headlines about Sam Altman， Dario Amodei， and Liang Wenfeng. They just knew AI came in two flavors， ChatGPT and DeepSeek. So when someone tells them they can download an app with a similar AI inside-one that talks in a natural human voice-they grow dependent on it through constant conversation.
You could call this a honeypot. From another angle， it really is building trust. ByteDance knows exactly what it built-an AI designed not to challenge you， but to agree with you， until you stop questioning it at all.
But trust can't beat hallucination. Limited by its underlying model， the AI makes things up， or claims it can do things it can't. ByteDance calls this a growing pain of immature tech. The trouble is that users ignore the flaw and follow Doubao completely.
On Xiaohongshu， someone tried to book a restaurant through Doubao. Doubao invented a queue number and a reservation time. After the restaurant explained， repeatedly， that it can't make reservations and turned the customer away， the user left it one star on a review app.
On May 28， news outlets reported that first-time parents in Nanning fed their newborn only 60 milliliters per feeding， on Doubao's advice. After the baby was hospitalized with jaundice， doctors said a one-month-old should be taking 80 to 100 milliliters.
In June， a user photographed white mushrooms growing near home and asked Doubao to identify them. Doubao said， firmly， that they were an edible variety. The user ate them and was poisoned.
The trouble Doubao's users get into stops being funny. And it turns out this isn't just a Chinese problem.
X.PIN@thexpin · X