Without the IT department's knowledge, sensitive data is being entered every day into AI tools like ChatGPT. The phenomenon is called Shadow AI, and it's one of the fastest-growing and most overlooked threats to data security today.
The IT landscape is changing, and we are seeing many devices that are adopting artificial intelligence in new ways. This brings entirely new possibilities for users, but also a challenge for the IT department, which now has to manage the systems. This is precisely where Shadow AI arises, when staff themselves find and use AI tools outside the school’s systems and without clear parameters. In this article, I look at the new IT landscape from the perspective of an educational institution and outline a possible direction for IT departments to build on.
The biggest challenge I see right now is an evolution of the “Bring your own device” issue, where employees bring their own devices. We can now call it “Bring your own AI”. We are seeing many staff members finding and using a range of, often insecure, AI tools without clear data processing agreements. This can be anything from the Chinese language model DeepSeek to deeply integrated AI tools on their own computers.
A major challenge is that the school’s policies are no longer sufficient, and responsibility for data security ends up with the individual staff member, who rarely has the prerequisites to assess the risk. This can involve anything from GDPR breaches to copyright infringement. For example, when a teacher uploads a whole textbook to an AI service to generate student questions, or when an administrative staff member uses ChatGPT to produce a clean copy of confidential minutes from management meetings involving sensitive HR matters. Such actions are often carried out with the best of intentions, but the consequences can be serious, both legally and for the institution’s credibility.
The problem becomes even greater when the individual staff member has their own computer on which they have chosen the programs themselves. Here, what is installed is highly changeable, and the IT department has no overview.
At the same time, we need to pay attention to what students do in lessons. They appear more experimental and less critical about which services they sign up for, and they do not always have a clear understanding of what constitutes sensitive personal data.
It can be hard to keep up, as everything seems to be augmented with AI in one way or another, but we must be proactive and find a new way forward.
The old IT policies are not built for the new reality, and therefore educational institutions and IT consortia need to establish a clear framework for AI. Unlike traditional IT policies, AI influences far more than we are used to, and therefore you need to set up a group with representatives from leadership, HR, legal, and those who will work with it day to day. Note that I do not think this is purely an IT task, nor is it a task to be parked in a corner of the organisation.
The group should jointly define what the educational institution wants to achieve with AI and, perhaps even more importantly, which data must under no circumstances be entered into public language models.
I believe the first step is to put in place four foundational building blocks:
This must of course take place within the framework of the GDPR, but the institution may also have additional data that require protection.
The challenge is that developments are moving very quickly. Apple Intelligence is being rolled out on Mac devices, Windows is integrating AI directly into the operating system, and the boundaries of when AI is used are becoming increasingly blurred. May an AI proofread an email before it is sent in Outlook? Such questions require clear answers before they arise in everyday practice.
As a school we can block various tools, but the challenge is that users will simply find alternatives. Therefore, the only sustainable solution is to offer a safe and approved alternative that also works well. However, there is a race between private companies and what a school is able to provide. Edu licences are often more restrictive, and some language models offered to schools today are second-tier AI systems.
By investing in an enterprise solution in which our data remain our own, such as Microsoft Copilot or Google Gemini for Workspace, we can give staff the tools they are asking for, but in a controlled and secure environment. When the approved solution is easily accessible and works well, the need to seek out insecure alternatives disappears.
The best defence against data leaks and mistakes around AI and IT is for staff to be trained and well informed in the new technology. The AI Act lays down a set of requirements regarding what individual staff members should be able to do if they interact with, or teach about and with, AI. Here I have summarised the key points:
This also means investing in training staff in the parameters for AI and ensuring they understand why we have these rules.
The above challenges with Shadow AI will not disappear by themselves, and it is not possible to ban them, as new systems keep emerging. Therefore, the only way forward is a three-pronged effort in which we set clear parameters in a coherent strategy, offer attractive and safe tools and, most importantly, invest in training our staff.
At the same time, I believe it is a task for the entire organisation to work with AI, because all staff will in one way or another be affected by it in their daily work. In this way, AI can become a genuine pedagogical opportunity for both the educational institution and the individual staff member, rather than a challenge that the staff member faces alone.
Only by doing all this can the educational institution move from firefighting to building a safe path forward, where artificial intelligence becomes a genuine educational benefit rather than an uncontrolled risk.
Author links open overlay panelNora McDonald a, Aditya Johri b, Areej Ali a, Aayushi Hingle Collier c
230388au.dk
Jonathan Villa
Birgitte Vestermark
Malenefw
Rikke Borup
Andrea Larnkjær
