Online privacy used to mean being careful about what you shared. You also had some control over what others shared about you, because much of that info was public. You could ask a friend to take down that unflattering photo on Facebook, for example.
Conversational artificial intelligence (AI) has changed that. Big AI companies may already know much more about you than you’re comfortable with, even if you’ve never used their services.
The problem goes beyond well-known products like ChatGPT, Gemini, or Claude. Many chatbots and companion AI apps can now remember past exchanges, personalize responses, use human-like voices and personas, and even initiate contact. The same features that make them useful also create privacy consent problems.
Information shared without consent
A recent YouGov survey shows 15% of adults in Australia have shared personal thoughts or feelings with a chatbot, and 11% disclosed something they never told anyone else.
The trend is even more pronounced for adolescent users. According to the Australian eSafety Commissioner, 54% of children aged 10-17 have used a chatbot for personal or social advice and one-third for life advice.
Because AI chat windows feel like private spaces for confiding, people also often share information about others — sometimes intimate details — without their knowledge or consent. A colleague might paste an email from you into ChatGPT to solve a problem. A friend might upload screenshots of your messages to make weekend plans. Your partner might discuss your latest quarrel. Your parents might share concerns about your health or finances.
You may never know. Yet information about non-users — people who haven’t even interacted with the chatbot — can still enter these AI systems through billions of everyday interactions. And we have no way of knowing what’s there.
Two sets of legal reforms
The issue is not just that AI companies know stuff about you, but what that knowledge might let them do behind closed doors. Information can be reused for training AI models, personalization, profiling, or advertising.
Intimate knowledge can be used to influence people, from pushing potentially harmful advertising to political persuasion or reinforcing extremist beliefs.
Australia is considering two sets of law reforms that could help regulate the privacy risks associated with chatbots, including for non-users.
Proposed privacy reforms would require personal information to be handled fairly and reasonably, and introduce a “right to erasure”. This would require large digital platforms to destroy personal information at an individual’s request.
Meanwhile, the proposed digital duty of care — an amendment to the Online Safety Act — would require online services to assess and manage foreseeable risks arising from their design and operation.
Together, these proposals get a lot right. Privacy reform can protect information about non-users, while the digital duty of care can address harms arising from how chatbots are designed and operated.
But how do you exercise a right to erasure if you don’t know who knows what about you? We need specific measures to ensure the legal requirements actually stick.
How to ensure the laws work
First, privacy should be built into the conversation, not buried in settings. Conversational AI can be designed to spot unnecessary sensitive info and discard it while still carrying out the directions in the user’s prompt.
Since the technical means to do so exist, providers of chatbots should therefore be required to identify and delete unnecessary information about users and third parties. They should also make it easy for users to review and delete stored information.
Second, using personal information to answer a prompt should not provide an indefinite blank cheque for its ongoing use. The AI might need info about another person to answer the immediate request. That doesn’t justify retaining it for training, profiling, targeting, advertising or unrelated personalization.
Third, erasure should not depend on having an account. Service providers need to offer practical ways for people to find out what info is held about them in the system, and request deletion.
On the other hand, such notification must also be safe. Automatically alerting everyone discussed could expose someone seeking help about domestic abuse, coercive control, whistleblowing, or confidential matters. The proposed digital duty of care could require platforms to manage these foreseeable risks.
Transparency is key
Finally, we should not have to take AI companies’ word for it. They should publish clear information about how they detect, retain and reuse third-party information, backed by independent testing and public reporting on whether safeguards actually work.
This matters particularly when providers profit from engagement, subscriptions, profiling and advertising. Australia’s proposed digital duty of care would allow the eSafety Commissioner to make rules that give independent researchers this kind of access. How well the rules work will depend on the details.
None of this requires banning conversational AI or stopping people from discussing their lives. What we need to do is push the responsibility upstream, to the providers of these tools.
People should be able to use conversational AI without having to manage the privacy risks it creates for everyone around them. The corporations that control and profit from these systems should not be allowed to treat such risks as other people’s problems.
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Follow us on X, Facebook, or Pinterest
