AI and Safeguarding: What NGOs Must Know Before Using AI

TL;DR:

• If your organisation works with children or survivors, agree a written AI rule now. Your staff may already be using these tools.
• The core rule is simple. No case details, names or identifying information about a child or survivor go into a public AI tool.
• Removing the name is often not enough. A combination of age, place, school and incident can identify a person in a small community.
• Safe uses exist: policy drafts, training with invented cases, general communications and spreadsheet help. Case notes, risk scoring and direct support to survivors should stay with people.
• Plan for mistakes. Staff need to know who to tell, and Kenyan law sets a 72-hour clock for notifying certain data breaches.

Who this guide is for

This guide is for safeguarding leads, programme managers, directors, data protection officers and board members. It applies to organisations in Kenya working with children, survivors of violence or trafficking, refugees and other people at risk.

You do not need technical knowledge. If you run a broader staff programme, read this with our guide on how to train NGO staff on AI. That guide covers training. This one covers safeguarding decisions, law and what to do when something goes wrong.

This article is general information and is not legal advice. For a specific case, speak to your lawyer or data protection officer.

Why does AI need its own safeguarding rule?

AI needs its own rule because pasting text into an outside tool can disclose it to a third party, and the output can be wrong in ways that harm a child. Tech Safety Canada, which supports anti-violence workers, describes AI use as a form of third-party disclosure. Data may leave your organisation, cross borders and be kept for an unknown time.

It also warns that banning AI outright tends to fail. AI is now built into everyday software, and staff may use it quietly with no guidance. A clear rule with clear boundaries is more likely to be followed than a blanket ban.

A real case: the child protection report in Victoria, Australia

The clearest public example comes from a child protection agency in Australia. In December 2023 the Department of Families, Fairness and Housing in Victoria reported a privacy incident to the state's information commissioner. A child protection worker had used ChatGPT while drafting a report for the Children's Court. The case involved a young child whose parents had been charged with sexual offences.

According to Information Age, the worker entered names and risk assessment information. ChatGPT's text then turned a doll that was part of the concern into a sign of good parenting, describing it as "age-appropriate toys". The commissioner's investigation looked at both the accuracy of the information and its unauthorised disclosure. Reports say the compliance notice required the department to block ChatGPT and similar tools for child protection workers.

That worker was trying to save time on a hard job. The lesson for your organisation is that good intentions and heavy workloads are exactly the conditions in which this happens.

What does Kenyan law say?

Kenya has no AI-specific law as of November 2026, but existing law already applies to what your staff type into AI tools. Here are the provisions a safeguarding lead should know.

Draft guidance to watch. The Office of the Data Protection Commissioner published a draft Guidance Note on Artificial Intelligence in July 2026, and consultation closed on 17 August. As of our check it remains a draft. Reports on the draft say that outside AI providers handling personal data would need a written data processing agreement, and that automated decisions significantly affecting children would need human oversight and parent or guardian involvement.

International standards. UNICEF's Guidance on AI and Children (version 3.0, December 2025) sets ten requirements for child-centred AI, including keeping children safe and protecting their data and privacy. If your funders use safeguarding standards, check how they apply to digital tools and whether your agreement with them says anything about AI.

What counts as identifying information?

Identifying information is anything that could point to a person, alone or combined with other details. The combination is what catches people out. In a small community or a known programme, a few ordinary facts are as good as a name.

Show your team examples like these. All of them are invented.

A useful test: could someone from that child's village, school or church work out who this is? If the honest answer is "maybe", it stays out.

What is safe, what needs approval and what should stay human?

Sort every use into one of three zones, based on the data and the effect on people. This table is a starting point. Your safeguarding lead should adapt it.

A 2026 UNICEF presentation on AI in gender-based violence work offers a similar spectrum. It places individual risk assessment and one-to-one support chatbots at high risk. Case management tools and data analysis sit at medium risk. Administrative tasks sit at low risk. The same presentation notes that GBV chatbots do not publish performance data, and that staff may skip human checks when they over-rely on a system.

Note one difference. Even a low-risk task such as translation becomes high risk when the text being translated is a survivor's statement. Always look at the data.

Why meeting note-takers and transcription need special care

Do not allow AI note-takers in case meetings, supervision or any conversation where a disclosure may happen. These tools record people who may never have agreed to it, and their errors can enter a permanent record.

In February 2026 the Ada Lovelace Institute published a study of AI transcription in UK social work, based on 39 interviews. The researchers did not formally test accuracy. Social workers did report real time savings. They also described errors: one said an AI summary wrongly indicated suicidal thoughts that the person had never raised. Reported checking time ranged from about two minutes to an hour, and there is no accepted standard for how long review should take.

A 2024 study of the Whisper speech-to-text system found about 1% of transcriptions contained whole phrases or sentences that were never spoken. In a child's case record, one invented sentence could do real harm.

Worked example: a donor report with no case data

Here is a safe way to use AI on a common task. Suppose a programme officer must write a quarterly narrative for a donor. All the material below is invented.

Step 1: prepare the input away from the AI tool. Use only totals and programme-level activity. Remove any small numbers and all stories about individuals.

Step 2: use this prompt.

I work for a Kenyan NGO that supports girls affected by violence. I am writing a quarterly narrative for a donor. Here are programme-level facts only: [number] life-skills sessions held; [number] staff trained in safeguarding; one new referral partner added; main challenge: long travel distances for families. Write a 300-word narrative in formal, respectful language. Do not invent any figures, quotes, names, locations or stories about beneficiaries. If information is missing, write [ADD DETAIL] so that a staff member can fill it in. Do not describe any individual's experience.

Step 3: add human content offline. If a story is needed, the safeguarding lead decides whether one can be used at all, with informed consent, and a person writes it.

Step 4: check it. Compare every figure with your records. Check that the AI has not added an emotional story.

What goes wrong. A tool may decline to help when it sees terms such as "sexual violence". Staff under pressure may then add real detail to explain themselves. Teach them to stop there and rewrite the request in general terms, or to write that section themselves.

Two more prompts to copy

Create a training scenario:

Create a fictional safeguarding training scenario for NGO staff in Kenya about a concern raised at a community event. Use invented names and an invented place. Do not include graphic detail. Give three discussion questions and the steps staff should follow under a typical reporting procedure: [paste your procedure, with no real cases].

Turn a policy into plain language:

Rewrite this safeguarding policy section in plain English for community volunteers with basic literacy. Keep every duty and timeline exact. Do not add new rules. Mark anything unclear with [CHECK]. Text: [paste the policy, with no case information].

A one-page AI and safeguarding rule you can adapt

Keep the rule short enough to fit on one page and read in a staff meeting. Adapt this draft with your safeguarding lead and data protection adviser.

Our AI and safeguarding rule

1. No names, case details, photos, voice notes or identifying information about a child, survivor or community member go into any AI tool, unless the safeguarding lead has approved that exact organisational tool and use in writing. Public AI tools are never approved for this.
2. "Identifying" includes combinations of age, place, school, family, incident, date or role.
3. AI is never used to decide who receives a service, to score a person's risk or to judge whether a concern is true.
4. AI does not talk to children or survivors on our behalf.
5. No AI note-takers or recorders in case meetings, supervision or safeguarding conversations.
6. Only tools on our approved list may be used for work. Personal accounts may not be used with work material.
7. A named person checks every AI-assisted document before it is used.
8. Never use AI to create or alter images of a child, including for fundraising.
9. If you make a mistake, report it to [named person, phone number] immediately. Reporting honestly will never be treated as misconduct.
10. Safeguarding concerns are reported through our normal procedure. AI is never a step in that procedure.

For the wider organisation, our guide to writing an AI usage policy sets out what else to include.

What should you do if someone pastes case details into an AI tool?

Act the same day, and treat it as a possible data breach and a safeguarding matter. Fast and honest reporting matters more than blame.

1. Stop. The person stops using that chat and reports it to the safeguarding lead at once.
2. Record. Note the tool, account type, time and exactly what was entered. Keep this record in your secure system, and avoid copying the details again.
3. Limit. Delete the chat if that is possible, and check the data settings. Deleting may not remove every copy, so do not assume that the risk has gone.
4. Assess. The safeguarding lead and data protection officer decide whether there is a real risk of harm to the person.
5. Notify if required. Section 43 of the Data Protection Act sets a 72-hour limit for notifying the Data Commissioner of qualifying breaches. Take legal advice quickly. Check your donor agreement as well.
6. Protect the person. Think about their safety first. Decide with specialist input whether and how they should be told.
7. Learn. Find out why it happened. Was the workload too heavy? Was there no approved tool? Fix that cause.

How should you choose an AI tool?

Choose a tool only after you know what data it will touch and what the supplier has promised in writing. Free consumer accounts and organisational accounts can be very different.

Two examples, as of November 2026:

• Google Gemini Apps: Google says that when Keep Activity is on, chats can be used to improve services, a subset is read by human reviewers, and reviewed chats may be kept for up to three years. Google also says human reviewers may still look at chats when it is off, for limited purposes such as safety.
• Microsoft Copilot and Copilot Chat for organisations: Microsoft says prompts and responses fall under its Data Protection Addendum, and are not used to train foundation models. Web search queries are handled under different terms, and controls vary by plan.

These commitments do not automatically make case data acceptable. They are only one part of the decision.

Ask each supplier:

• [ ] Is our data used to train models?
• [ ] Can staff at the supplier read our data, and when?
• [ ] Where is the data stored and processed? Is it outside Kenya?
• [ ] How long is it kept, and can we delete it?
• [ ] Will you sign a written data processing agreement?
• [ ] Which sub-processors are involved?
• [ ] Can we turn off web search, memory and third-party add-ons?
• [ ] How will you notify us of a breach, and how quickly?

For the most sensitive drafting, some organisations look at open-source AI that runs on their own computers. This keeps text in the building, but it does not remove your data protection duties or the risk of wrong output.

The new harm your safeguarding policy may not cover

AI has created new forms of abuse, so update your safeguarding policy as well as your office rules. The Internet Watch Foundation reports that in 2025 it assessed 8,029 AI-generated images and videos as showing realistic child sexual abuse. Girls made up 97% of the illegal AI-generated images it assessed. It also reports that photos of real children can be used to make such material.

Tech Safety Canada lists further AI-enabled abuse, including sextortion, impersonation and fabricated evidence.

Four practical steps:

• Review children's photos on your website and social media. Consider using fewer clear, named images of children, and review your consent process.
• Add AI-generated abuse to your safeguarding policy and your reporting procedure.
• Train staff on how to respond when a child reports a fake image. Believe them, avoid forwarding the image and follow your procedure.
• Never "test" whether a tool can make such images. Attempting it could be a serious crime and is harmful in itself.

If a child needs help, Childline Kenya is free and open 24 hours on 116. The national gender-based violence helpline is 1195.

Are you ready? A checklist for directors and boards

Before staff use AI for work, you should be able to tick each of these.

• [ ] We know which AI tools staff are already using. We asked without blame.
• [ ] The safeguarding lead helped write our AI rule.
• [ ] Every staff member and volunteer has read and discussed the rule.
• [ ] We have a short approved tools list.
• [ ] Any tool touching personal data has been assessed before use.
• [ ] Our consent forms and privacy notice match what we actually do.
• [ ] Staff know who to tell after a mistake, including at weekends.
• [ ] Our safeguarding policy covers AI-generated abuse.
• [ ] Partners and contractors who handle our data follow the same rule.
• [ ] We review this every six months. The tools change fast.

Do not forget the people doing this work. Caseworkers carry heavy loads, and that is often why they reach for AI. UNICEF's presentation warns of mental overload and pressure to trust a system over human judgement. Giving staff safe ways to reduce paperwork is part of safeguarding them.

Questions people ask

Can our NGO use ChatGPT at all?

Yes, for work that does not involve case or personal information, such as drafting a general policy or preparing training with invented cases. Set the rule first, and make sure a person checks every result.

Is it enough to remove the child's name?

No. Age, location, school, family details and incident details can identify a child when combined. Leave out those details, or use a fully invented example.

Can we use AI to summarise case notes if we pay for a business account?

A paid account may give better contract terms, but it does not settle the question. You would need a legal basis, a data protection impact assessment, a written agreement with the supplier, safeguarding approval and a plan for checking errors. For most organisations, the answer today should be no.

Can a chatbot give advice to survivors?

We advise against it unless specialists have tested it thoroughly and a person is available at all times. A UNICEF presentation in 2026 placed these chatbots in its high-risk group. A wrong answer or a monitored phone could put someone in danger.

Do we need parental consent before using AI with children's data?

Section 33 of the Data Protection Act generally requires a parent's or guardian's consent. A body that exclusively provides counselling or child protection services may not need it. The best-interests requirement and other duties continue either way, so take legal advice.

Can AI translate a survivor's statement?

Avoid public tools for this. Translation sends the full text to the tool, and errors can change the meaning of testimony. Use a trained interpreter or translator who is bound by confidentiality.

Sources

All links last checked 9 November 2026. We could not open the Data Commissioner's own PDF of the draft AI guidance on that date, so we have relied on the reports listed.

• Data Protection Act, No. 24 of 2019, Kenya Gazette copy on the ODPC website
• Children Act, No. 29 of 2022
• Computer Misuse and Cybercrimes Act 2018, section 24
• Digital Policy Alert, ODPC draft Guidance Note on Artificial Intelligence
• PPC Land, report on the ODPC draft AI guidance
• Office of the Victorian Information Commissioner, investigation into the use of ChatGPT by a child protection worker
• Information Age, Australian Computer Society (25 September 2024)
• IDM, "Victorian agency breaches privacy through use of ChatGPT" (26 September 2024)
• Tech Safety Canada, "Using AI Safely in Survivor Advocacy" (2026)
• UNICEF presentation, "AI in GBV Programming: Perils and Potentials", Learning Network forum (2026)
• UNICEF Innocenti, Guidance on AI and children, version 3.0 (December 2025)
• Ada Lovelace Institute, "Scribe and prejudice?" (11 February 2026)
• Koenecke et al., "Careless Whisper: Speech-to-Text Hallucination Harms", arXiv (2024)
• Internet Watch Foundation, AI and child sexual abuse imagery
• Google, Gemini Apps Privacy Hub
• Microsoft Learn, Enterprise data protection in Microsoft Copilot and Copilot Chat
• Child Helpline International, Childline Kenya
• United Nations in Kenya, GBV helpline 1195

Where FuKazee fits in

FuKazee trains NGO teams to use AI for the writing and reporting in front of them, with data protection taught first. We work with invented cases and sample documents, so staff never practise on real records. We show the free version of each tool, and we are honest about where AI should not be used at all.

We have done this work with Maisha Girls Safe House, a Nairobi NGO supporting girls who are survivors of sexual violence. Thirteen staff took part on 18 and 19 September 2026. The rules on what must never go into an AI tool were taught on the first morning, before anyone opened a tool. All exercises used fictional or sanitised material.

Training will not replace a safeguarding policy, a data protection adviser or a good supervisor. It can give your staff a shared rule and safe habits. See how it works on our page for AI training for NGOs in Kenya.

To talk through training that respects your safeguarding duties, email us at lead@fukazee.com.