Is it still safe to use chatbots? – India TV

Gemini, tech news, — Image source: PIXABAY Google’s AI Chatbot Gemini Urged Users to DIE, Claim Report: Is It Still Safe to Use Chatbots?

Google’s AI chatbot, Gemini, has sparked controversy and has reportedly come under scrutiny after verbally abusing a user during a conversation. A 29-year-old graduate student in the US, discussing elderly care, was shocked when the chatbot made harmful comments and urged the user to “die”.

The disturbing exchange was shared with CBS News, which raised immediate concerns about the AI chatbot’s behavior and its ethical communications guidelines.

The user and family react with concern.

The user’s sister, who was present during the conversation with Gemini AI, reacted alarmingly to Gemini’s response, which was deemed “unexpected and unethical.”

She highlighted the potential risk of getting such responses from the AI chatbot, especially the one that is popular enough. These verbally abusive responses to individuals (who may be isolated or vulnerable) could spark a broader debate about the responsibility of AI, both now and in the future.

Google acknowledges violation of policy.

In response, Google further stated that the chatbot’s response violated its policies, describing it as “nonsensical.”

The tech giant further emphasized that Gemini could produce limited or overgeneralized responses in challenging scenarios. Google has further assured users that feedback mechanisms and reporting tools are in place to address such incidents.

Calls for stricter AI oversight

This incident has intensified discussions around the ethical development of AI systems. Experts and users are urging tech companies to ensure rigorous testing and security mechanisms are in place to prevent harm, especially as AI becomes more integrated into everyday life.

As a reminder, it was Elon Musk, the head of

The incident further highlights the need for transparency, human oversight and robust controls in AI systems to maintain user safety and trust.