Using AI safely with customer data in New Zealand

If you run a business, you may have wondered whether you can put your customer details into ChatGPT. It depends on the tool and on what you put in.

The Privacy Act 2020

In New Zealand, the Privacy Act 2020 governs how organisations collect, use, store and share personal information. Personal information is anything that identifies a person: names, phone numbers, addresses, emails, account details, photos and notes about them. The Act applies to businesses of every size, including a two-person trade business, and it applies whether the information is on paper, in a spreadsheet or in an AI tool.

The Act sets out information privacy principles. The Commissioner updates its guidance, so check it is current. These are the principles that matter most once AI is involved.

• Collect only what you need, and tell people why. If you're going to use their details in a new way, think about whether they'd expect it.
• Keep it safe. You need reasonable safeguards against loss, misuse and unauthorised access. Sending customer details to a tool you don't understand sits badly with this.
• People can ask to see their information and have it corrected. That gets harder to answer if it ends up scattered across tools.
• Check it's accurate before you use it. AI can produce things that sound right and are wrong, so don't act on AI output about a person without checking.
• Don't keep it longer than you need to. Old data you no longer use is a risk with no benefit.
• Use it for the purpose you collected it for, and be careful about disclosing it to others. Putting it into a third-party tool may count as disclosing it, so check how the tool handles it.
• Be careful about sending it overseas. Many AI services run on servers outside New Zealand, and there are rules about disclosing personal information to overseas parties. Ask where your data goes.

If a privacy breach causes serious harm, or is likely to, you have to notify the Privacy Commissioner and the people affected. If you think that has happened, get proper advice promptly.

What not to paste into public AI tools

In this article, "public tool" means a free or personal AI account. Terms differ between providers and plans and they change, so I won't make claims about what any particular product does with your input. A safe habit is to put in nothing you'd be uncomfortable seeing leave the business.

Put in nothing you'd be uncomfortable seeing leave the business.

• Customer names combined with contact details, addresses or account information.
• Health information, or anything sensitive about a person's circumstances.
• Staff records, performance notes, pay details or disciplinary matters.
• Bank details, passwords, API keys and login links.
• Confidential contracts, pricing agreements, or a customer's drawings and designs.
• Anything a customer told you in confidence.

Safer habits

• Remove identifying details first. Replace "Jane Smith, 12 Example Street" with "Customer A" before asking for help with a letter, then put the details back yourself.
• Use a business plan with clear data settings, and read what it says about how your input is used.
• Write a one-page rule for your team covering what's fine, what isn't and who to ask.
• Keep a list of the tools you use, so you can answer questions about where information went.

Questions to ask a supplier

If you're buying an AI tool or hiring someone to build one, whether it's us or anyone else, these questions are fair, and a good supplier will answer them without fuss.

1. Where is our data stored and processed, and does it leave New Zealand?
2. Is our data used to train the AI models? Can that be switched off, and is it off by default?
3. Who at your company, and which sub-suppliers, can see our data?
4. How long do you keep it, and how do we get it deleted when we leave?
5. What happens if there's a security incident? How and when would we be told?
6. Can we export our data in a usable form?
7. How do we handle a request from a customer to see or correct their information?
8. Is all of this written into the agreement?

If a supplier can't say where your data goes, think hard before using them.

Keeping a person in charge

Privacy covers decisions as well as where data sits. AI can draft, sort and suggest. It shouldn't have the final word on something that affects a person, such as refusing a service, chasing a debt or making an assumption about someone.

• A person reads AI-drafted messages before they go to customers, at least until you trust the process.
• A person checks facts about individuals before acting on them.
• A customer can always reach a human. An AI receptionist should say it's an assistant and hand over when asked.
• Someone in the business is named as responsible for how AI is used. The Privacy Act requires organisations to have at least one privacy officer, and in a small business that's often the owner.

When we build something, a person stays in charge and the output is checked. Sensitive data gets extra care. When we design an AI chatbot or assistant, the first question is what information it will see and what it needs. The AI receptionist demo deals with customer enquiries, and the same questions apply to it.

A starting checklist

• List the AI tools anyone in the business uses today.
• Write a short rule on what must not be pasted into them.
• Check the data settings on each paid account.
• Ask the supplier questions above before adding any new tool.
• Decide who is responsible, and tell the team.
• Review the list once or twice a year, because tools and terms change.

This site has its own privacy policy. If you're unsure whether you can do this yourself, our article on AI consultant or DIY may help.

Next step: the privacy notice sets out how this website handles enquiry details, and the FAQ covers how project data is agreed.