AI Chatbots at Work

AI Chatbots at Work: The Privacy Nightmare Your Business Probably Hasn't Thought About Yet

Let's Talk About What's Really Going On Here

Okay, so AI chatbots are everywhere now. ChatGPT, Claude, Gemini, Character.AI, Replika... you name it. Your employees are probably using them. Heck, you're probably using them. And look, I get it—they're incredibly helpful. But here's the thing nobody's really talking about: there's some genuinely concerning stuff happening behind the scenes.

Recent research out of Stanford, plus work from various safety groups, has uncovered something pretty alarming. Six major U.S. AI companies are taking what you type into their chatbots and using it to train their models. And I'm not talking about some conspiracy theory here—this is documented, confirmed stuff. For small and medium-sized businesses? This could be a massive problem you didn't even know you had.

So let's dig into this, because I think you need to know what's actually at stake.

What Exactly Are These "Companion" Chatbots Anyway?

Think of them as AI tools that don't just answer one-off questions. They're designed to have ongoing conversations, remember what you've talked about before, and create this sort of... relationship with users. Unlike those basic customer service bots that follow scripts, these things learn from every interaction.

Now here's where it gets dicey for businesses. When your employees use these tools—maybe to troubleshoot code, maybe to draft an email, sometimes just to chat during lunch—they might be accidentally handing over your company's secrets to third parties. Proprietary info, client data, trade secrets... the works.

Jennifer King from Stanford's Human-Centered Artificial Intelligence research center put it pretty bluntly: six leading AI companies are feeding user conversations right back into their training models. Which means that discussion your sales manager had with ChatGPT about your pricing strategy? Yeah, that could theoretically end up as part of the AI's knowledge base.

Kind of makes you think twice about what you've been typing in there, doesn't it?

Wait, They're Actually Using Our Conversations to Train Their AIs?

Short answer: Yep, most of them are.

Unless you've specifically opted out (assuming that option even exists), or you're paying for some fancy enterprise version with different terms.

King's research is pretty clear on this. When asked if people should worry about their privacy using these chatbots, she basically said "absolutely yes." Let me break down what we know:

What Each Platform Is Actually Doing With Your Data

ChatGPT: OpenAI might use your chats for training unless you dig into the settings and opt out. The free version? Not much privacy protection there. The enterprise version? That's a different story, but it costs money.

Claude: Here's something interesting—Anthropic actually changed their terms of service to make conversation training the default. You have to actively opt out now. A lot of people (and businesses) probably haven't noticed this shift yet.

Google Gemini: Same deal. Whatever sensitive stuff you're sharing with Gemini could end up training the model.

The real kicker: Every company's doing it differently. Some let you opt out, some don't. Some are upfront about it, others... not so much. Try creating a consistent security policy around that mess.

Okay, But What Could Actually Go Wrong?

Let me paint you some scenarios, because I think this stuff becomes real when you see the actual risks.

Your Intellectual Property Walking Out the Digital Door

Say one of your developers is stuck on a bug. They copy-paste a chunk of your proprietary code into ChatGPT to get help debugging it. That code could:

Get absorbed into the AI's training data
Sit on some company's servers forever
Potentially pop up when someone else asks about similar problems
Be exposed if that AI company gets hacked

I'm not saying it will happen. But it could. And "could" is a problem when we're talking about your competitive advantage.

The Compliance Nightmare

If you're in a regulated industry, this gets even messier:

Healthcare providers sharing patient info = HIPAA violations
EU customer data going into these systems = GDPR problems
Financial data exposure = FINRA compliance issues
Client information handled outside approved systems = SOC 2 failures

Each one of those could mean serious fines. And frankly, "I didn't know our employees were using ChatGPT" probably isn't going to fly as a defense.

The Productivity Question

Now, this might sound strange, but... some research shows these AI chatbots are becoming really popular with teenagers and young adults. Which means your younger employees might be, uh, having personal conversations with AI during work hours. Not exactly the biggest concern here, but worth noting.

If You Employ Anyone Under 18

This one's trickier. Dr. Nina Vasan from Stanford Brainstorm has called the youth safety issues around AI companions a potential "public mental health crisis." Her recommendation? Kids shouldn't be using these tools, period. Not until there are better safeguards.

So if you've got minors working for you and they're accessing AI tools on company devices... you might want to think about your liability there. What's your duty of care as an employer?

What Are the Experts Actually Saying?

Look, I'm not trying to be alarmist here, but the expert consensus isn't exactly reassuring. Stanford's research basically says these chatbots are "failing the most basic tests of child safety and psychological ethics." And the privacy protections? Inadequate across the board.

Dr. Vasan doesn't pull her punches—she's talking about this potentially reaching public health crisis levels. Granted, she's focused mainly on young users, but the underlying concerns apply to workplace settings too. We're talking about AI systems that form emotional connections with users, remember personal details, and potentially manipulate behavior.

Jennifer King's take on privacy is even simpler: Should you worry? "Absolutely yes." The inconsistent policies, the default opt-in for training, the lack of transparency... businesses can't even properly assess their risk exposure with all this uncertainty.

So What the Heck Are We Supposed to Do About This?

Alright, let's get practical. Here's a game plan you can actually implement.

Your 30-Day Action Plan (Because Tomorrow Never Comes)

Week 1: Figure Out What You're Dealing With

First, you need to know where you stand. Survey your team—who's using these tools? Which ones? For what? Don't make it feel like an interrogation... just get a sense of the landscape. Then draft an Acceptable Use Policy. Nothing fancy, just clear guidelines about AI tools.

Week 2: Put Up Some Guardrails

Time for technical controls. Block non-approved AI sites on your company network. Use mobile device management to stop people from installing these apps on company phones. Set up some data loss prevention rules—basically, systems that flag when someone's copying sensitive info to a web browser.

I know this sounds like a lot. But honestly? It's not as complicated as it sounds, and it's way easier than dealing with a data breach later.

Week 3: Talk to Your People

This is the part most businesses skip, and it's a mistake. Mandatory training on AI chatbot risks. Give people actual examples—"Don't paste client names," "Don't share our pricing structure," that kind of thing.

And here's the key: explain why. People are way more likely to follow rules when they understand the reasoning. "Because I said so" doesn't work with adults.

Week 4: Look at the Fancy Versions

Investigate enterprise AI solutions. Microsoft Copilot with commercial data protection. Google Workspace AI with business-grade security. These cost money, sure, but they come with actual privacy controls. Whether you need them depends on your specific situation... but at least know what's out there.

Long-Term: Building Something Sustainable

Adopt a Zero-Trust Mindset

Here's my philosophy: assume anything you put into a third-party AI system is compromised. Forever. Train your team accordingly:

Use anonymized examples instead of real client data
Describe problems conceptually instead of pasting actual code
Keep financial information and strategic plans out entirely

Create Your Approved List

Make a whitelist of AI tools that meet your standards. Enterprise agreements, clear data policies, proper security certifications... whatever matters for your industry. Then communicate clearly: these are approved, everything else isn't.

Stay Vigilant

This stuff changes constantly. Anthropic's recent policy shift proves that. You need regular audits, policy updates, and someone keeping an eye on new developments.

Maybe Get Some Help

Look, managing all this might be overwhelming. Especially if you're a small business without a huge IT team. That's where a Managed Service Provider can be worth every penny. They can monitor your network, maintain security controls, handle employee training, manage enterprise AI deployments... basically take this entire headache off your plate.

Should We Just Ban AI Chatbots Completely?

Honestly? I don't think blanket bans work. They're hard to enforce, and you might be putting yourself at a competitive disadvantage. AI tools—when used properly—can genuinely boost productivity.

My take: ban unapproved AI chatbots on company systems, but provide secure alternatives. Focus on education over prohibition. Make it about smart usage, not zero usage.

The goal isn't to stop innovation. It's to make sure innovation happens safely.

What About Personal Devices Though?

Ugh, this one's tough. You can't control what people do on their personal phones at home. But you can:

Set crystal-clear expectations in your employee handbook. Company confidential info stays confidential, regardless of what device you're using. Explain the stakes—their job security, the company's survival, client trust. When people understand the "why," compliance goes up dramatically.

Give them legitimate alternatives. If they've got access to secure, approved AI tools for work tasks, they won't feel tempted to use consumer versions.

Add it to contracts. Confidentiality agreements, employment contracts... make AI chatbot usage an explicit part of these documents.

And maybe consider threat intelligence services that scan for your proprietary info appearing in datasets or training data. Think of it as an early warning system.

The Bottom Line (Let's Be Real Here)

Your data's being used: Six major companies are feeding conversations into their training, with inconsistent ways to opt out.
Privacy protections are weak: Every expert agrees current measures aren't good enough for business use.
You can't ignore this: Data breaches through AI chatbots could mean regulatory fines, client lawsuits, and losing your competitive edge.
Education matters more than you think: Most exposure happens through well-meaning employees who just don't know the risks.
Technical controls aren't optional: Policies alone won't cut it.
Enterprise solutions exist: They cost more than the free versions, but you get actual security controls.
Professional help is available: If this feels overwhelming, MSPs exist specifically to handle this kind of thing.

Time to Actually Do Something

The AI landscape is changing fast. New tools launch every month. Existing platforms change their policies constantly. What was true six months ago might not be true anymore—Anthropic's policy shift proves that.

Small and medium businesses are in a weird spot. You need AI tools to stay competitive, but you don't have the massive IT security teams that big corporations deploy.

The answer isn't to ignore AI or ban it entirely. It's to be strategic. Clear policies, appropriate controls, ongoing education. Whether you handle it internally or get help from an MSP, the key is recognizing that this isn't something you can just set up once and forget about.

So here's what I'd suggest: Start your 30-day plan this week. Survey your current AI usage. Draft some policies. Have those conversations with your team.

Because the data your employees are sharing with AI chatbots today? That could be tomorrow's breach headline. But only if you don't act.

On second thought, maybe start today, not next week. Why wait?

Sources

King, Jennifer. "Be Careful What You Tell Your AI Chatbot." Stanford Human-Centered Artificial Intelligence, Stanford University, 2024.

Transparency Coalition. "Complete Guide to AI Companion Chatbots." Transparency Coalition, 2024.

A quick note: This article covers a complex, evolving topic. Policies and practices at AI companies change frequently. Always verify current terms of service and privacy policies before making decisions for your business.

< Older Post

Newer Post >

Why Your Accounting Firm's IT Infrastructure Isn't Just a Technical Problem—It's a Business Lifeline

May 27, 2026

Why Your Accounting Firm's IT Infrastructure Isn't Just a Technical Problem—It's a Business Lifeline The Real Cost of "We'll Do Better" Tax season waits for no one. Neither do cybercriminals. That's the reality facing accounting firms today. You're managing sensitive financial data, client information, and compliance obligations—while operating infrastructure that may be one breach away from disaster. Yet many firms find themselves trapped in a cycle: their current IT provider promises improvements, quarter after quarter, but nothing fundamentally changes. Sound familiar? Three Vulnerabilities That Keep You Up at Night 1. The Backup That Doesn't Exist When You Need It Backups are supposed to be your safety net. But a backup that fails silently is worse than no backup at all—because you don't know you're exposed until it's too late. When we assess accounting firms, we consistently find backup systems that haven't been tested in months. No restoration practice. No disaster recovery plan. Just hope. 2. The Old Hardware Ticking Time Bomb Servers beyond five years old aren't just aging—they're becoming liability. Parts become unavailable. Warranties expire. And when failure happens during tax season, you're not calling Dell. You're searching eBay for replacement components and praying they work. 3. The Compliance Gap Nobody's Talking About HIPAA. GDPR. FINRA. PCI. Each regulation has specific requirements—and many require 100% compliance, not 99%. You could be meeting 19 out of 20 requirements and still be technically non-compliant. That one missing item? It's the one the auditor finds. Or worse—the one a cybercriminal exploits. Why Accountants Are the #1 Target Here's what cybercriminals know: accounting firms have access to money, client data, and predictable workflows. They don't need to break into your system dramatically. They just need to: Watch your email for payment instructions and client data transfers Intercept wire transfer requests by impersonating leadership Deploy ransomware during your busiest season when downtime costs the most Compromise your clients through your systems, making it your liability One firm we worked with experienced a ransomware attack that started with an employee reconnecting an infected old laptop. It spread to three machines before monitoring stopped it. The result? Incident response. Notifications. Regulatory scrutiny. A breach that could have been prevented. The Partnership Approach That Actually Works Here's what separates a true IT partner from a vendor: Understanding Your Business Rhythm : Your IT infrastructure shouldn't be a generic setup. It should reflect the reality of tax season—when you need everything stable, secure, and running flawlessly. That means proactive maintenance in January. Quarterly checkups. Hardware refreshes on a schedule, not a crisis. Risk Aversion Built Into Every Decision : You're risk-averse for good reason. Your clients depend on you. A system outage doesn't just cost you money—it costs them. A data breach damages trust that takes years to rebuild. A true partner approaches IT with the same mentality: prevent problems, not just fix them. Compliance as a Roadmap, Not a Checkbox : Your risk assessment should give you a clear picture: Where are you compliant? Where are you vulnerable? What's the priority order to fix gaps? And critically—which compliance requirements actually apply to your specific business? (Not every regulation is equally relevant to every firm.) Treating You Like Family, Not a Ticket Number : When you become a customer, you're no longer a support case. You become someone they're invested in protecting. That means they know your team. They understand your processes. They're proactive about calling you with concerns instead of waiting for things to break. The Questions to Ask Your Current Provider When was your backup last tested and restored to a clean environment? What's your timeline for replacing servers over five years old? Can you show me a compliance assessment with specific gaps and remediation steps? How do you prevent business email compromise attacks? What's your incident response plan if we get breached? If they can't answer these clearly—or if they're giving you the same vague promises they gave you last year—it's time to look elsewhere. Your Next Step The difference between accounting firms that sleep well at night and those who worry about the next disaster often comes down to one decision: choosing a true partner over a service provider. If you're ready to move from crossed fingers to actual security, let's talk about what a proactive, risk-aware IT partnership looks like for your firm. Your clients deserve better. So do you.