Six steps applied to a hypothetical adaptive gradin platform.
There is a specific compliance requirement in the EU AI Act that most edtech teams don’t mention, and that I myself didn’t fully understand until I analyzed it in detail.
Article 27 requires those who implement high-risk AI systems (which includes schools and public educational institutions that use AI for assessment, admission, or student monitoring) to conduct a fundamental rights impact assessment before implementation. This isn’t a simple, generic data protection impact assessment, but a rights impact assessment, specifically focused on the effect of the AI system on the fundamental rights of the people it affects.
When I looked for a practical example of how this applies to an educational AI system, I found framework documents, regulatory summaries, and compliance checklists. What I didn’t find was a tutorial: someone who applied the methodology to a specific type of edtech system and presented the results.
So I created one, using a hypothetical adaptive assessment platform as a test case. This article documents that exercise. I completed it as part of a course on Fundamental Rights Impact Assessments (FRIA) for generative AI projects, and I applied the methodology to the educational technology context I’m familiar with. It’s important to note that I haven’t conducted a formal FRIA in a production environment. What I can offer is the methodology applied to a realistic scenario, with the gaps and uncertainties clearly visible.
This last point is important. FRIAs that seem straightforward often indicate that the questions weren’t formulated rigorously enough.
What is a Fundamental Rights Impact Assessment (FRIA) and When is it Required?
A process that examines possible effects on human rights when individuals encounter or face outcomes from artificial intelligence defines what a Fundamental Rights Impact Assessment entails. Though designed to review consequences, it focuses specifically on those tied to core freedoms and dignity. Its structure helps identify risks before deployment, guiding decisions through organized evaluation rather than assumptions. When systems influence lives, such assessments offer clarity about potential harm. They do not guarantee safety but highlight where interference might occur.
Before launching high-risk AI systems, public authorities must carry out a FRIA under Article 27 of the EU AI Act. Where such technology is used by private groups in sectors serving societal needs, the same rule applies. Operation cannot begin until this assessment is completed.
Recording this evaluation happens through an EU system created by Article 71. It operates separately, yet alongside, the Data Protection Impact Assessment required by the GDPR. Although DPIAs examine dangers tied to handling personal information, FRIAs take a wider view - looking at effects on various rights. These include access to learning, fairness across groups, protections for young people, and ways individuals can seek justice when needed.
Although in practice they are often conducted together, the conceptual scope differs: a DPIA asks, “What are the risks of this data processing?”, while a FRIA asks, “Which fundamental rights does this system affect, and what are the risks to each of them?”.
It matters since artificial intelligence may impact basic rights without directly processing data - such as generating outcomes that disproportionately hurt specific populations or removing human judgment from choices affecting individual freedoms, despite ethically managed information practices.
The System I Used as a Test Case
A hypothetical adaptive assessment platform forms the basis of this task (I’ve named it AdaptLearn), intended for use in a secondary school setting. Functioning dynamically, the system adjusts content based on student performance. One key feature tailors question difficulty in real time. Another tracks individual progress across subjects throughout the term. Besides these, it offers feedback summaries to both students and teachers after each session.
Grading begins as soon as a student submits their work. A language-based system assesses how ideas are organized, whether reasoning holds up, if sentences follow standard rules, and if facts check out. The score is generated automatically, with results ranging from zero to one hundred, and the feedback appears in simplified written format, tailored to each response. The whole process runs without human input once started.
Next comes the part where past test results shape future lessons. Based on those combined scores, a number is calculated to guide students toward suitable tasks. When someone keeps falling short of the mark, they are directed to simpler topics. Reaching beyond the cutoff opens doors to tougher challenges. The path adjusts quietly behind the scenes.
Looking closer, instructors receive access to a performance overview that displays both personal and group-level data. Some learners appear highlighted when signs suggest they might struggle academically. A built-in alert system draws attention where needed most.
This system describes real tools currently in operation. Though it operates in practice, its function meets several criteria for high risk according to Annex III. Automated assessment of student performance forms one such concern. Another lies in how it routes learners based on perceived difficulty levels. Because of these features, deployment in public schools demands a FRIA first.
The Fria Approach In Practice
Step 1. Describe the system purpose and who it affects
Before identifying rights impacts, a clear description is needed of what the system does, who operates it, and whom it affects—not in marketing jargon, but in operational terms specific enough to support the risk analysis.
For AdaptLearn, the operational description might be: an AI-powered content assessment and guidance system, deployed in secondary schools, operated by teachers, and affecting students aged 11–18. The system processes students’ written work and behavioral performance data to generate grades, feedback, guidance decisions, and risk indicators. Teachers have access to the dashboard, and students, on the other hand, can see their grades and feedback, but not their guidance scores or risk indicators.
That last sentence—that students don’t see their guidance scores or risk indicators—is the kind of detail revealed in the description. It’s not obvious in a general product description, but it’s directly relevant to the rights analysis.
Among those affected by this system are students whose work is evaluated and whose educational paths are influenced, teachers who rely on dashboard information to make pedagogical decisions, and, indirectly, parents who receive information about their children’s progress without necessarily understanding its origin in artificial intelligence.
Step 2. Pinpoint Key Rights Involved
The EU Charter of Fundamental Rights provides the framework. For AdaptLearn, the relevant rights include several specific to the educational context.
-
The right to education, enshrined in Article 14 of the Charter, is directly at stake: system routing decisions influence the educational content a student can access, and systematic errors in such routing constitute an interference with the right to education.
-
The right to non-discrimination, enshrined in Article 21, is relevant because any systematic variation in grading accuracy between demographic groups constitutes potential discriminatory treatment.
-
Children’s rights, enshrined in Article 24 (including the best interests of the child as a paramount consideration), apply because the individuals affected are minors.
-
The right to an effective remedy, enshrined in Article 47, is relevant because students affected by erroneous AI decisions need a functional mechanism to challenge and correct such decisions.
-
Furthermore, data protection, enshrined in Article 8, applies to the processing of student performance data.
-
Privacy, enshrined in Article 7, is affected by behavioral monitoring functions.
-
It is worth highlighting the freedom of thought contemplated in Article 10: a system that grades argumentative essays using a linguistic model trained on a specific corpus can subtly disadvantage reasoning patterns that deviate from the model’s baseline, constituting an interference, albeit less obvious, with intellectual freedom of expression.
These are seven fundamental rights. A FRIA that only covers data protection and non-discrimination leaves important aspects of the rights landscape unaddressed.
Step 3. Assess Likelihood and Severity of Impact for Each Right
For each right, it is necessary to evaluate two points: the probability that the system’s operation will interfere with that right and, if so, the severity of that interference. Furthermore, severity must consider reversibility (can the damage be repaired?), scope (how many people are affected?), and vulnerability (are the affected groups particularly exposed?).
Let’s see this point applied to AdaptLearn:
-
The right to education faces a medium-to-high probability of impact due to incorrect routing decisions. A student assigned basic content because the grading model doesn’t suit their writing style may spend weeks or months with material below their actual level, with effects that will accumulate throughout their academic career. This harm is partially reversible if the teacher detects and corrects it, but not if the control panel reinforces the model’s evaluation instead of requesting a human review. Here, we can consider the severity high for the individuals affected, but moderate in terms of scope.
-
Non-discrimination faces a medium to high probability of impact if the grading model functions differently among student groups defined by their linguistic background, writing style, or cultural references. In a diverse classroom, this is not a theoretical borderline case. The severity depends on the magnitude of the achievement gap, which requires empirical measurement.
-
Regarding the right to an effective remedy, students are generally unaware that their educational path is being determined by an AI routing score, and therefore cannot challenge what they are unaware of. The probability of an effective remedy being unavailable in practice is high; the severity is high for any student whose educational progress is harmed by an error they have no mechanisms to detect.
The child’s rights, as defined in Article 24, are implicated at all times. The question posed by the best interests of the child standard is not whether the system is generally beneficial, but whether this specific implementation, with its specific error modes and monitoring mechanisms, serves the best interests of each affected child.
Step 4. Existing Safeguards Identified
This step documents the measures the implementing organization has in place to mitigate the identified risks. This is where FRIA assessments start to feel uncomfortable, as the exercise forces you to name what already exists and evaluate whether it is truly sufficient.
For the AdaptLearn project implemented in a typical secondary school, existing safeguards might include: a teacher review before grades are finalized, a general GDPR privacy notice provided to parents during enrollment, and a grade appeals process that students can initiate through standard school procedures.
In light of the rights risks identified earlier, these safeguards are, at best, partial. Teacher grade review is a safeguard for the right to education and the right to effective redress, but only if teachers conduct a critical review rather than simply ratifying the AI’s results. The privacy notice covers data protection rights but says nothing about the AI’s routing score or risk alert function. The appeals process assumes that students know what to appeal; if they are unaware of the routing score, they cannot access the appeals process.
The objective of this exercise is precisely to explicitly identify this deficiency: that protective measures exist, but they do not adequately cover the identified risks. The FRIA should make this clear, as this deficiency is what drives the corrective action agenda.
Step 5. Identify Residual Risks
Residual risks are those impacts on rights that persist even after considering existing safeguards. In the case of AdaptLearn, the main residual risks are:
Students from demographic groups for whom the grading model performs less well will consistently receive lower grades and be placed at lower content levels. Teachers may reinforce this situation rather than correct it, as the dashboard presents the AI’s results as reliable. This therefore represents a risk of discrimination and a violation of the right to education, and none of the existing safeguards described above adequately address it.
Students will be unaware that their educational path is being determined by an AI-assigned score, meaning that the right to effective redress is not available in practice, even if a formal appeals process exists. This gap persists even after considering current safeguards.
Teachers have nominal oversight but lack the information or framework necessary to exercise effective supervision. They only see the dashboard, and they don’t see the model’s confidence levels, the performance data of the subgroups, or the documentation of known system limitations. Monitoring that appears functional from the outside can be largely illusory.
Step 6. Document Mitigation Measures and Assign Responsibility
This step transforms a Risk Impact Analysis (RIA) from an analytical exercise into a practical governance document. For each residual risk, there must be a specific mitigation measure, a responsible party, and a timeline.
Risk of Discrimination and Routing
The school implementing the system must require the educational technology provider to disclose disaggregated performance data before implementation (specifically, grading accuracy and routing accuracy broken down by relevant student subgroups). If the provider cannot provide this data, this constitutes sufficient grounds to block implementation.
- Responsible party: The school’s data protection officer and the head of the relevant department.
- Timeline: Before implementation.
Right to Effective Remedy
The school must inform students and parents that there is an AI-generated routing score that influences access to content, explain in plain language how it works, and create a specific mechanism for students or parents to request a human review of a particular routing decision.
- Responsible: School administration.
- Timeline: Before implementation, as part of enrollment information.
Teacher oversight
The vendor must provide training on system limitations, documentation of known failure modes, and guidance on when to override AI results. The school should integrate this into initial teacher training and make it a requirement for using the system.
- Responsible: Shared between the vendor and the school.
- Timeline: Before implementation, ongoing.
Where FRIAs Fall Short in Real Use
These analyses can have certain limitations depending on how they are used, specifically whether the organization conducting the FRIA has the legitimacy to act on its findings.
A common mistake is conducting the FRIA after the implementation decision has already been made. In this case, the FRIA becomes an exercise in retrospective justification rather than a prospective risk assessment.
Another mistake is assigning the FRIA to a compliance function that lacks specialized knowledge. Assessing the impact on rights in an educational context requires someone who understands how educational systems operate.
A third mistake is treating the FRIA as a one-off document rather than a dynamic record. Although the text does not specify a review frequency, conducting an FRIA already implies ongoing monitoring. A system assessed before implementation can develop new risk profiles as the student population changes, the model evolves, or usage patterns shift.
FRIA Completeness Checklist
Before filing a Fundamental Rights Impact Assessment for a high-risk educational AI system:
[ ] System description is specific enough to support risk analysis (not marketing language)
[ ] All affected persons identified: students, teachers, parents, other indirect stakeholders
[ ] Rights inventory covers at minimum: right to education, non-discrimination, privacy, data protection, rights of the child, right to effective remedy
[ ] Likelihood and severity assessed separately for each right, with justification
[ ] Existing safeguards documented with honest evaluation of their adequacy
[ ] Residual risks named explicitly — risks that safeguards do not adequately address
[ ] Mitigation measures assigned to named owners with timelines
[ ] Provider-side obligations identified (performance data, training documentation, known limitations)
[ ] FRIA conducted before deployment decision is finalized, not after
[ ] Pedagogical domain expertise included in the assessment, not just legal/compliance
[ ] Review process defined for ongoing monitoring
[ ] Document registered in EU database per Article 71
Frequently Asked Questions
Is a FRIA Different from a DPIA?
While these processes share common ground in schools and universities, they serve separate purposes. Not every data review covers fairness in learning access, yet each must consider individual liberties. One stems from privacy law, targeting how information gets used. The other emerges from artificial intelligence rules, looking at wider societal effects (such as fair treatment, legal protection, or childhood safeguards). Often, teams handle both assessments at once due to similar groundwork. Still, their starting conditions differ, along with what they cover and whether authorities need notification.
Does a private school have to conduct a FRIA?
It could be argued that Article 27, in its present form, covers public institutions along with private actors carrying out duties assigned by such institutions. Whether a privately run school qualifies hinges largely on national classifications and if it serves a role deemed to benefit the wider community. In many cases, especially where state funds are involved or oversight follows official education systems, treating the rule as applicable makes practical sense. Even without clear legal mandates, evaluating compliance can still align with sound management practices and help preserve institutional credibility. High expectations from stakeholders often make proactive review a reasonable course.
What happens if the FRIA reveals risks that cannot be adequately mitigated?
Should deployment continue unchecked, consequences may follow. If flaws appear but no changes occur, problems grow likely. When a risk review flags major concerns yet pushes ahead regardless (fixes missing) the report stops being protective. It becomes something else: evidence of neglect. Early warnings exist so teams can act while solutions remain possible.
Who Should Conduct the FRIA?
A deployment group must carry out the FRIA, yet its effectiveness hinges on participant roles. While legal or compliance staff bring understanding of regulations, operational insight into the AI system comes from technical personnel. Input from education experts, preferably teachers, adds necessary context about learning environments. Without such inclusion, assessments led solely by compliance often overlook critical impacts. Who joins shapes what gets seen.
What level of detail should the FRIA include?
A national market surveillance authority expects thoroughness when reviewing compliance. This becomes the benchmark in practice. Vague risk descriptions fail to meet it, even if safeguards appear listed alongside. Without judging how well those protections work, confidence drops off. Assigning responsibility for each countermeasure matters just as much as naming them. Skipping ownership details weakens the whole account. Registration in the EU database demands precision of another order entirely. Comparisons across implementations depend on such detail. Oversight loses value when specifics go missing.
What The Exercise Showed
Surprisingly, engaging with the FRIA framework (applied here to a theoretical setup, treated more as practice than strict adherence) revealed insights not clearly expected at the start.
One surprise came from counting the core rights touched by an average school-based AI tool. Instead of just privacy and bias concerns, the picture expanded: pulling in learning access, young user safeguards, mental autonomy, plus avenues for redress when things go wrong. Each layer carried its own kind of exposure, separate from typical data-handling dangers.
What matters next is how fragile protections can be when real-world details shift. Teacher oversight supports fair grades (provided the scrutiny stays sharp). A chance to challenge outcomes offers recourse, assuming learners understand their rights. The FRIA demands proof that defenses function outside theory, not just promises tucked in policy documents.
Later came issues around when to act. Steps like demanding detailed performance reports, enforcing transparency in score routing, or tying launch approval to staff preparation could only work if enforced early. Once agreements were sealed, such influence faded sharply. An evaluation done afterward still holds value, yet its role in shaping outcomes weakens considerably.
Because of that timing issue, FRIAs should happen sooner rather than later in procurement. Not doing them early creates more problems down the line.
This piece forms part two of the series titled “AI Governance from the Ground Up”. Earlier, we looked at ethical concerns around student use of AI chatbots in learning environments. Coming up: an exploration into how AI literacy becomes a mandatory requirement within schools, along with specific demands it places on educational technology systems.
This is being written while I’m learning. Should you notice mistakes in the method or rules review, please point them out.
Some information may be outdated
