Across the UK, schools and colleges are adding AI chatbots to their websites and parent portals, and it’s very easy to understand why. It’s a bot that answers admissions queries at 9 pm, handles the same enrolment questions for the hundredth time, and takes pressure off what is usually a stretched front office.
Furthermore, most of these deployments deliver on their promises; however, when it comes to cybersecurity, they can fall short for two reasons.
Firstly, educational institutions assume that the chatbot has been rigorously tested to ensure that it is indeed safe to use. Some have, but many haven’t. Secondly, educational institutions add the chatbot to their websites and portals without checking and double-checking that the setup is secure. It doesn’t matter how secure the chatbot is if the setup hasn’t been done securely.
In this article, we will explore exactly how to assess whether a chatbot is secure and which common risks to look out for to keep your students and staff safe.
What Are AI Chatbots?
An AI chatbot is a piece of software that uses artificial intelligence to understand a question written in plain language and reply with a relevant answer.
Most of the tools appearing on school websites today are built on large language models, which is the same underlying technology behind assistants like ChatGPT, and this is what lets them handle a question they have never seen before rather than only the exact phrases someone programmed in advance.
That flexibility is what sets them apart from the older, rule-based chatbots that simply matched a few set keywords to canned replies. More importantly still, it is also what makes their security worth thinking about, because a system that can interpret free-form language, and that is often wired up to other systems to fetch its answers, has a far wider range of ways it can be misused.
We cover chatbot security more generally here, but the rest of this article focuses on what that means for schools.
Why Schools Are Deploying AI Chatbots
Before we get into the common risks and how to assess them, it’s useful to understand a little bit more about why schools are deploying AI chatbots. As we discussed in the intro, the case for automation is fairly obvious, especially if you’ve worked at school.
A chatbot that answers routine questions about term dates and uniform policy saves staff hours every week. During enrolment, when the same forty questions arrive in slightly different words, that value multiplies. Using a chatbot for basic parent communications, signposting, and a first pass before a human gets involved is a sensible use and one that is happening in almost all industries.
The concern is the difference between a bot that answers public questions and one that has been handed the keys to records, payment flows or safeguarding information without anyone checking the connection is secure.
AI Chatbot Security Is a Shared Responsibility
When AWS, Azure and Google Cloud went mainstream, plenty of organisations treated them as black boxes that were secure by default. All you had to do was upload your data, flip a switch, and you were good to go.
The reality was a shared-responsibility model in which the provider secured the physical infrastructure and platform, while the customer remained on the hook for access control, configuration, and the data itself.
For years, the gap between those two things showed up as publicly exposed storage buckets leaking records that should never have left the building. In many of those cases, the platforms themselves held up, and it was the human configuration that let them down.
Now, AI deployments are following a very similar curve where the model vendor invests heavily in safety, but that investment says nothing about how your specific chatbot was wired up, what it can reach, or who can talk to it.
The Security Risks of AI Chatbots in Schools
Anyone who works at a school knows that cybersecurity is a frontier that schools have to take very seriously. According to the government’s Cyber Security Breaches Survey, 73% of secondary schools, 88% of further education colleges, and 98% of higher education institutions had identified breaches or attacks in the last 12 months. We have written more about where these settings are most exposed in our guide to the 10 biggest cybersecurity gaps in UK schools.
With attacks on educational institutions so common, introducing an AI chatbot to the mix will only increase your organisation’s attack surface, and so it’s almost inevitable that a cyber criminal will attempt to abuse it.
So, let’s look at the risks that could come with that.
Data Segregation Failures: One User Reading Another’s Data
Many chatbots are connected to a knowledge base so they can give specific answers, much like your mouth and ears are connected to your brain. If that connection is not scoped correctly per user and per session, the bot can return information that belongs to someone else.
In a school, that might mean a parent finding another family’s contact details or fee status, or a child’s record appearing where it should not. A system that retrieves documents without proper access controls will happily fetch whatever it can see, and it can usually see a lot more than it should.
Token Cost Abuse: Someone Running Up Your Bill
While many AI chatbots are implemented as a means of saving money, something that is usually very high on any educational organisation’s agenda, if misconfigured, they can cost you thousands.
Basically, AI services are charged by usage, so every question and answer costs money in tokens. If the integration is left open, say an endpoint with no authentication or an API key sitting in the page source, an outsider can fire thousands of requests at it.
That means your paid integration becomes a free service for them and a nasty invoice for you, and at scale, the same abuse can knock the bot offline for the people who actually need it.
Prompt Injection and Chatbot Misuse
Chatbots take their instructions in plain language, which means they can be talked into things. A carefully worded message can override the original instructions, coax the bot into revealing its setup, or extract data from the systems it connects to. In effect, this is a form of social engineering, aimed at the bot rather than at a member of staff.
In a school, the stakes are obvious because a bot tricked into producing inappropriate content or exposing internal information is not a comfortable conversation to have with parents or the ICO.
When Is an AI Chatbot Safe to Deploy in a School?
This is not an argument against automation because, when configured correctly, the benefits of an AI chatbot far outweigh the negatives. Really, all this comes down to understanding what your chatbot has access to and whether it has been configured correctly to prevent misuse.
A chatbot that answers public, low-sensitivity questions, has no access to personal data, and is subject to sensible usage limits, is likely to be perfectly safe. The moment it connects to records, payments, or anything covered by data protection law, it deserves the same scrutiny you would give any other system that holds that data.
That principle sits at the heat of school cybersecurity more broadly, which we cover in our headteacher’s guide to protecting your community.
It’s always important to remember that “secure by default” is a marketing phrase, not a deployment status, and shouldn’t be taken as gospel.
How to Assess Your School’s AI Chatbot Security
If your school or trust already runs an AI chatbot, or is about to, the useful question is a plain one: what can it actually access, and who can actually reach it?
Fortifi Cyber Security runs AI security assessments built specifically for this. We map what your chatbot is connected to, test how it responds to misuse, confirm that a single user cannot pull another’s data, and verify that your integration cannot be hijacked to run up costs.
Frequently Asked Questions
Are AI chatbots safe for schools to use?
They can be, provided they are set up with care. A chatbot that only answers public, low-sensitivity questions and has no access to personal data is usually perfectly safe. The risk rises sharply the moment it is connected to records, payments or safeguarding information, at which point it deserves the same scrutiny as any other system holding that data.
What kind of data could an AI chatbot expose?
If the connection to its knowledge base is not scoped correctly for each user and session, a chatbot can return information belonging to someone else, such as another family’s contact details or fee status, or a pupil’s record. It can only expose what it has been given access to, which is why limiting that access matters so much.
What is prompt injection?
Prompt injection is where someone uses carefully worded input to override a chatbot’s original instructions. It can be used to coax the bot into revealing its setup, producing inappropriate content, or extracting data from the systems it connects to. Because chatbots take their instructions in plain language, they can be talked into things a more rigid system never would.
Does using a chatbot from a well-known provider make it secure by default?
No. The model vendor invests heavily in the safety of the underlying platform, but that says nothing about how your specific chatbot has been configured, what it can reach, or who can talk to it. As with cloud storage, security is a shared responsibility, and the configuration is your side of that bargain. Treating a chatbot as secure straight out of the box is exactly the mistake to avoid.
How can we check whether our school’s chatbot is secure?
The most reliable way is an AI security assessment, which maps what the chatbot is connected to, tests how it responds to misuse, confirms that one user cannot pull another’s data, and verifies that the integration cannot be hijacked to run up costs. If you already run a chatbot, or are about to, that is the moment to ask what it can actually access and who can reach it.