Privacy Considerations with AI Tools

Artificial intelligence (AI) tools come in all sorts of flavors. There are notetaking and transcription tools, chatbots, device-wide agentic AI features, grammar and writing tools, translation assistants, AI summaries, and research tools, among others. As a term, “AI” may refer to features in apps, apps themselves, third-party plug-ins, or a tool that’s built right into your operating system . It’s a mess of acronyms and data flows that’s incredibly difficult to keep track of, which makes understanding the privacy practices of these tools challenging.

Because of the variety of uses of the term, for this guide, we’re going to use “AI” for anything that’s marketed as such by the software companies that release the tools. While different software implements AI features in different ways, they all should trigger the same sorts of questions before using them for any sort of information you want to keep private. 

Most AI tools require some sort of privacy sacrifice, so understanding the risk is important. But these risks apply differently depending on your needs at the moment. For example, there’s less risk to using an AI tool to get a summary of a TV show’s previous season because you forgot what happened, but there may be some risk if you’re using a tool to help collate research documenting human rights abuses. That does not mean you should never use AI for higher risk activities–just that you should be aware of the privacy and security risks of doing so, and carefully weigh those risks against the benefits of using the tool. There’s no one-size-fits all recommendation for implementing (or avoiding) these tools. Instead, you have to consider what potential risks they may create for you, and decide if those risks are worth the trade off. 

This guide is an overview of the many considerations to make before choosing to use an AI tool or software with AI features. EFF does not endorse or recommend any specific tools.

First: Figure Out If You’re Already Using AI Tools #

AI is everywhere, and just figuring out if some tool or app you’ve been using for years is now implementing some sort of AI feature can be a journey on its own. For example, if you’ve been using the grammar checker Grammarly integration for the past decade, you might not have noticed the slow process of it turning into an AI-first tool that collects and stores much of what you type while using it. Similarly, Adobe Acrobat, once a humble PDF reader, now has all sorts of AI features that makes it so you may inadvertently send the content of private PDF documents onto cloud services. 

Take some time to reassess and audit all the software you’re using for any sort of sensitive work or other projects. Make a list of all the software you use, then go through and investigate whether it has added AI tools, whether the software provides ways to disable those features, and try to learn where the data might go if you use those AI features (we’ll get into this more below). If you’re ever not sure, search online for the tool name and “AI” to try to learn as much as you can about whatever implementations it might have.

While you’re investigating your current software, take special care to investigate any add-ons for that software. This can include something like the Grammarly extension for Chrome and Firefox, or third-party integrations in software, like Perplexity for MS Office. Look for features that feel like they could be potentially problematic to your privacy concerns. For example, an AI notetaker that’s transcribing and summarizing every video meeting might pose specific risks to journalists and activists.

These smaller features are easy to lose track of, and will likely have different privacy practices than whatever core software you’re using. The good news is that most software has a straightforward way to look for these sorts of add-ons—somewhere in the Settings of an app, you’ll find a section called “Integrations,” “Linked Apps,” “Add-ons,” or something similar that allows you to revoke access to these third-party tools. In some cases, these third-party integrations may be referred to as “Model Context Protocol” or MCP, which is a standard for connecting AI applications into other software, like giving the AI tool Claude Code access to a notetaking app like Notion. Take stock of what you find and make sure you’re still comfortable with these sorts of third-party integrations.

It’s good practice to audit your extensions, add-ons, or other integrations frequently. Not just for AI, but for all the software you use.

Understanding the Tool’s Privacy Practices #

Trying to read through privacy policies or understand how AI tools do or do not protect user privacy can be extremely difficult. But instead of spending hours going blurry eyed trying to read a privacy policy, you can start to get a grasp on some of the company’s data practices by first understanding what different AI tools are capable of, what the industry standards around data storage are, and what sorts of potential risks exist you may want to consider. 

Protect your privacy as best you can with chatbots by:
• Using accountless or “guest” modes whenever possible
• If you do need an account, be sure to use a unique password and two-factor authentication when available
• Investigate the chatbots privacy settings, and disable any data sharing settings you can
• Opt out of training whenever you can
• Consider the risks and benefits of sharing any private info with a chatbot, and try to minimize the amount of personal info you share
• For chatbots that have ads, turn off ads personalization

The Difference Between Cloud and Local AI Tools #

Most AI tools operate in the “cloud,” or, in other words, are “online-only.” This means everything you do with that AI tool may be collected, stored, and potentially accessed by the company that runs it. There are also tools that operate only on your device, and others still that are a hybrid, sometimes operating locally and other times operating online. It can be dizzying to figure out.

Local

Because a local model is running on your computer, it tends to be the most privacy protecting way to use an AI tool, though it does tend to require some technical know-how to set up and a reasonably modern computer to run. 

In some cases, a “local” AI is software that you’ve installed yourself and likely control how it works, often through an “AI client” that’s essentially software that can run different AI models. Some examples of AI clients include KoboldCpp, LM Studio, Ollama, or Llamafile. These sorts of tools allow you to run models on your own computer, where you can then use it however you see fit, like for speech to text, grammar checking, or whatever else you dream up. 

Other local options include purpose-built software that runs locally, but doesn’t require you to find and choose a model yourself. Examples of this sort of AI tool include the transcription tool OATS or the chat and image tool Off Grid AI Desktop.

You should also be aware that even if a model is running locally, if it is configured to run other tools on your computer or on the web (for instance, doing web searches), your data can be leaked to third parties through those tools. 

Just because the software runs locally doesn’t mean there will not be any risk. As with any tool, take the time to research the developers who make it, research any controversies, and learn how the software works before using it.

Cloud-based

The majority of AI tools that most people will interact with are cloud-based. This means that most of the AI processing happens on someone else’s computer. This tends to be clear when you go to, say, a website to log into a chatbot, but can also be true for some AI tools built into desktop or mobile software. For example, Microsoft’s Copilot may be included in Word, Excel, or Powerpoint’s desktop apps, but if you engage with Copilot in those apps, the AI processing is likely done online, meaning whatever you asked it to do will be done on the company’s servers. This can mean that the documents, text, and other information you’re providing will end up on the company’s servers, where it may be stored and may be used by the company for training (more on this below).

Cloud-based AI tools promise different levels of privacy, but any AI tool that operates online will carry some potential privacy risk. For example, some chatbots may promise privacy through various methods, like not logging user interactions, not sharing data with other companies, or by attempting to create technical barriers so the company cannot access chat history. These sorts of limitations may work for you in some security plans, but not in others, so take the time to research any privacy promises and look online for any cases where those promises may not have been kept.

Hybrid

Then we have AI tools that take more of a hybrid approach, where they remain on device when they can, but outsource to the cloud for more complicated requests. This is the sort of setup you’ll find in most commercial consumer hardware, like Apple’s Siri that it packs into its desktop and mobile operating systems, Google’s Gemini on Pixel phones, and others. 

It can be very difficult to figure out what data is kept locally and what data may be sent to the cloud for processing. Look into the company’s documentation to try to get a better idea of what AI features remain on the device. If you still can’t figure it out, try disabling the internet connection or your device then using the AI feature. If it still works, it’s likely an on device feature.

Some AI tools may list one or several “certifications,” like SOC, ISO, or HIPAA compliancy. These certifications or audits can be useful to get an understanding of how the company handles data for enterprise users, but doesn’t always apply to consumer-offerings. Be sure to still research the tools’ privacy practices. 

What “Temporary” and “Private” Modes Really Deliver #

Some chatbots offer “private modes” (sometimes called “incognito,” “temporary,” or something similar). These modes tend to promise that they will not use your chats for AI training, and, if the chatbot supports history, these questions won’t be stored and used in that history. But that doesn’t mean the company doesn’t store that data at all. For example, at the time of publication, Google Gemini’s Temporary Chat feature keeps those chats for 72 hours. ChatGPT keeps them for 30 days. So does Claude. These may change, so be sure to check what the current timing is.

Because this varies by chatbot, and will likely shift over time as terms and technologies change, it’s worth investigating how long the chatbot you’re using stores information for. Because the data is still stored online, it is best not to think of it as private.

Many chatbots and other similar AI tools offer a way to share the contents or results of a chat with others through links. Be aware that these 'share chat' features on AI assistants often create a public link, potentially including your name alongside the chat content. Avoid sharing sensitive conversations this way. We’ve seen several instances where these chats ended up in search engine results.

“Don’t Use This to Train AI” Does Not Equal “Cannot Access Data” #

The more information an LLM-based AI tool trains on, the more supposedly accurate and useful it can get, so companies have an incentive to collect as much data as possible to train their AI tools. By default, many AI tools collect data on how you interact with them, then use that to attempt to make their tools better. When this happens, the data you’re providing gets ingested into the training data set, which may mean that, in some select cases, the data could be recoverable by someone using the trained model in the future. 

It may also be subject to “human review,” a process that most AI companies use where a person manually reviews whatever an AI produces to check for accuracy. Once your data is used to train an AI model, it’s not typically possible to remove its influence on that model. 

Depending on the type of tool you’re using, this can mean the company behind that tool can collect a variety of information, ranging from every query you type into a chatbot to the full text of a document or the entirety of a transcribed meeting. How that data gets used, or how it might get combed through for personal information, varies by platform too, but OpenAI’s explainer is indicative of the process:

We retain certain data from your interactions with us, but we take steps to reduce the amount of personal information in our training datasets before they are used to improve and train our models. This data helps us better understand user needs and preferences, allowing our model to become more efficient over time.

Note the phrase, “reduce the amount of personal information,” which makes clear that it may not eliminate the personal information. 

In most cases, you can ask the company not to use your data for training its AI. Look in the settings for privacy options like, “Do not train on my content,” or “Help improve,” or “Improve the model,” or “AI data retention.” There are a lot of AI companies out there, but here is the documentation for disabling training on the current major platforms:

• ChatGPT
• Perplexity
• Gemini
• Claude
• Copilot

However, even without training, the company may hold onto that data for other reasons. For example, chatbots may offer “history” functions, or a transcription company will likely save whatever you had transcribed until you delete it. 

The data stored by an AI company is therefore subject to the same sorts of risks as any other cloud-based software company: data breaches and the potential for government requests (more on this below). When it comes to protecting against data breaches, there is a limit to what you can do, but if the AI tool offers security measures like two-factor authentication, be sure to enable it alongside a unique password.

How to Think About Law Enforcement and Government Requests #

Any data stored by a tech company may be useful to law enforcement or governments and that’s just as true for AI tools as anything else. This could include your chat history, generated notes, transcriptions, basic subscriber information or identifiers, and most other interactions you have with these tools. Even if this data is not “public,” as in it’s not posted to a social media feed or shared widely online, law enforcement can often get access through a subpoena or search warrant.

We know companies do receive requests for this information because we’ve seen transcripts of chats appear in court cases. Likewise, some AI companies, including OpenAI and Anthropic, and others, publish transparency reports that detail how many requests they get.

When law enforcement goes looking for information, they may go to the company who makes a chatbot, the one that makes a meeting transcription tool, or the one that makes that email summarizer, just like they would any other tech company. 

It’s important to be mindful of how much data AI tools collect and store. But because AI tools can be as straightforward as a chatbot that’s standalone software, or a little more complex, like a summarization feature or transcription feature that’s integrated into another tool, it can be very difficult to know what company collects that data and what their policies are. If your security plan includes law enforcement access as a concern, consider what, if any, impacts that AI tool may have to your risk profile.

Risks with “Agentic AI” Tools #

Agentic AI—the type of software that typically combines LLMs with external tools to autonomously carry out multi-step tasks, like researching and then purchasing plane tickets—poses significant and different sorts of privacy risks than AI tools that are limited to one specific application.

In order to function, many of these so-called “AI agents” need complete and total access to your device. That can mean the software may interact with otherwise secure and private tools, like an end-to-end encrypted chat app, or an offline notes application. This poses a privacy risk to anything stored on those sorts of applications. 

Similarly, many of the agentic tools that are designed to browse the web, shop for you, or book travel may require other sorts of personal information to be useful, like name and address, email, calendar, credit card numbers, and more. As with any collection of this information, there is always the risk of data breach or abuse. Agentic tools may also have significant security issues themselves, as we saw with OpenClaw. Because of this, agentic AI has the potential to erode any anonymity or pseudonymization practices you may employ, and may upload information you do not intend.

Be aware of the potential risks with any agentic tool, or any AI tool that has access to everything on your computer or device. It can be difficult to set up guardrails around any data you want to keep absolutely private.

Questions to Ask Yourself Before Using an AI Tool #

Choosing an AI tool isn’t all that different from any other tool, nor from security planning in general. But to help get you started, here are some questions to think about when you’re considering whether to implement an AI tool into your work:

• How transparent is the company who makes it about what it can and cannot access?
• What is the worst that could happen with my data if the company is compromised?
• Has the AI tool been subject to privacy or security criticisms online and in news stories?
• Am I certain that I am downloading the correct app or browser extension?
• Are there access permissions that make me uncomfortable, like accessing my email or messages?
• Am I comfortable with the privacy practices? Is it clear to me how the data will be used?