Customer trust in AI is conditional. A 2025 Twilio study found that 51% of consumers are uncomfortable sharing personal or financial information with AI agents, and 66% are uneasy about AI having access to their full history with a business. Salesforce's 2024 research showed that only 42% of customers trust businesses to use AI ethically — down from 58% in 2023.
At the same time, consumers want the convenience AI provides. BCG found that more than 60% express high trust in generative AI results during purchase journeys. The pattern is clear: customers accept AI for convenience and efficiency, but they want transparency, restraint, and control when it comes to their personal data.
For small businesses deploying AI customer service, this creates a practical challenge: how do you capture the benefits of AI while respecting customer privacy and maintaining trust? This guide provides the answers — practically, not legally. (For specific legal advice, consult a qualified professional.)
What Data Does AI Customer Service Typically Collect?
AI customer service agents typically process several categories of data during conversations.
Customer identifiers: Names, phone numbers, email addresses — collected when the AI captures leads or books appointments.
Order and account data: Order numbers, booking references, account status — used when answering queries about orders, deliveries, or appointments.
Conversation transcripts: The full text of the customer's messages and the AI's responses — stored for quality review, training, and handoff context.
Knowledge base content: Your business information (pricing, policies, products) — used by the AI to generate responses.
Metadata: Timestamps, channel used (WhatsApp, web chat, Telegram), conversation duration, resolution status — used for analytics and performance monitoring.
The important distinction: most SME AI platforms do not collect data beyond what the customer provides in the conversation. The AI is not scraping social media profiles, accessing browsing history, or connecting to external databases — unless specifically configured to do so.
What Should You Tell Customers?
Transparency is the single most effective trust-building measure. SurveyMonkey's 2026 data shows that 14% of consumers would lose trust in a business if they interacted with an AI agent that did not clearly disclose it was AI.
Best practice: Three disclosures.
At the start of the conversation: "Hi, I'm an AI assistant for [Business Name]. I can help with [topics]. For anything else, I'll connect you with our team." This sets expectations and establishes trust.
When collecting personal information: "I'd like to collect your name and phone number so our team can follow up. This information is used only for your enquiry." This provides purpose limitation — the customer knows why you are asking.
Accessible privacy notice: Link to a plain-language privacy notice from your chat entry point or website. This does not need to be a 20-page legal document — a clear paragraph covering what you collect, why, how long you keep it, and how customers can request deletion is sufficient for most SMEs.
Key Compliance Frameworks for SMEs
GDPR (EU/UK)
If you serve customers in the EU or UK, GDPR applies. The practical requirements for AI customer service: identify a lawful basis for processing data (legitimate interest is typically appropriate for customer service). Conduct a Data Protection Impact Assessment if processing is likely high-risk. Ensure transparency about AI use. Honour data subject rights (access, correction, deletion). Document your data processing activities.
The UK ICO specifically states that organisations must identify a lawful basis before sharing personal data and that a DPIA is required when processing is likely to result in high risk.
PDPO (Hong Kong)
Hong Kong's Personal Data (Privacy) Ordinance applies to any business operating in Hong Kong. The PCPD's 2024 Model Personal Data Protection Framework for AI provides practical guidance: establish AI strategy and governance, ensure procurement governance for AI tools, provide staff training and awareness, conduct risk assessments, and maintain human oversight.
The PCPD has separately emphasised the importance of creating internal AI policies — especially to control employee use of generative AI and reduce privacy risks.
Practical Compliance Checklist for SMEs
Tell customers they are talking to AI. Explain what the AI can and cannot do. Only ask for information necessary for the task. Avoid collecting sensitive personal data (health, financial, identity documents) through AI chat. Mask or redirect payment and ID data where possible. Offer an easy human alternative for sensitive matters. Keep a plain-language privacy notice linked from your chat entry point. Know how to handle data deletion requests. Review your AI vendor's data handling policies.
How to Choose an AI Vendor with Privacy in Mind
Before committing to any AI platform, ask these questions.
Where is customer data stored? Reputable vendors will specify the data centre location and jurisdiction. This matters because data protection laws vary by country.
Does the vendor use my customer data to train its AI models? Most SME-focused platforms do not. Ask for written confirmation.
Can I delete customer data on request? This is a legal requirement under GDPR and good practice everywhere. The vendor should support data deletion at the account and individual customer level.
Who can access conversation logs? Understand who within the vendor's organisation can see your customer conversations. Look for role-based access controls.
What happens to data if I cancel my subscription? Your customer data should be deleted or returned when you leave the platform. Get this in writing.
Omago, an AI agent platform that helps SMEs automate customer conversations across WhatsApp, Telegram, and web chat, provides clear data handling documentation for customers evaluating privacy compliance. For Hong Kong businesses subject to PDPO requirements, this transparency is particularly important.
Frequently Asked Questions
Do I need a privacy policy specifically for AI customer service?
You do not need a separate policy, but your existing privacy notice should cover AI-assisted communication. Add a brief section explaining that customer enquiries may be handled by an AI assistant, what data is collected during these conversations, and how customers can opt for human-only communication.
Can AI customer service comply with GDPR?
Yes. GDPR does not prohibit AI in customer service — it requires transparency, lawful basis, data minimisation, and respect for data subject rights. An AI agent that discloses its nature, collects only necessary data, and provides human alternatives for complex matters is GDPR-compatible.
What if a customer asks to delete their data?
You should be able to honour this request. Check whether your AI platform supports individual conversation deletion and customer data removal. Under GDPR, you have 30 days to respond to a deletion request. Under PDPO, you must comply with data access and correction requests.
Is it safe to collect customer phone numbers through AI chat?
Yes, for the purpose of follow-up communication. Phone numbers are personal data and should be stored securely, used only for the stated purpose, and deletable on request. Do not collect phone numbers unless needed — if the customer's question can be answered in the chat, collecting a phone number is unnecessary.
What about recording conversations for training?
If you use conversation transcripts to improve your AI's knowledge base, disclose this to customers. Under GDPR, using conversation data for AI improvement may require a separate lawful basis or consent. Under PDPO, ensure that data is only used for the purpose for which it was collected, or obtain consent for additional uses.
Sources: Twilio "Inside the Conversational AI Revolution" (2025), Salesforce "State of the AI Connected Customer" (2024), SurveyMonkey "Customer Service Trends" (2026), BCG Consumer Trust Research, UK ICO AI Guidance, EDPB AI and Data Protection Guidance (2024–2025), PCPD Model Personal Data Protection Framework for AI (2024).