GenAI Alone Isn’t Enough—Pair It with Learning Analytics.
by Manuel Valle
Professor Martinez was excited when her university deployed an AI tutor for first-year differential equations. Students logged hundreds of hours with the system, they felt confident of their skills, and the AI provided detailed step-by-step solutions to every problem. Homework was delivered on time, and quality was better than ever. Three months later, the midterm exam told a different story: 40% of students failed. When she reviewed their work, the pattern was clear: students could reproduce the AI's solution steps but couldn't explain the underlying concepts or adapt to problems with slightly different framing. The AI had been used, but had learning actually occurred?
Deploying AI without a feedback mechanism is like navigating without a compass. This exploration reveals the critical role of Learning Analytics in ensuring AI-driven education is effective, equitable, and truly intelligent. Without it, we risk investing in powerful tools whose impact we cannot see or steer.
What is Generative AI in Education?
Generative AI (GenAI) refers to algorithms that can create new content, including text, images, and code. In education, this technology powers tools like personalized tutors that can explain complex topics, automated assistants that generate lesson plans and quizzes, and creative platforms that help students brainstorm and write. Its strength lies in its ability to produce novel, context-aware material on demand.
What is Learning Analytics?
Learning Analytics (LA) is the process of measuring, collecting, analyzing, and reporting data about learners and their interactions. The goal is to understand and optimize the learning process. It answers critical questions: Are students engaged? Where are they struggling? Is this teaching method effective? LA provides the evidence-based insights needed to make informed educational decisions.
The Dangers of Flying Blind: GenAI Without Analytics
Deploying Generative AI without a Learning Analytics framework creates significant risks that can undermine educational goals. It leaves educators unable to verify effectiveness, address inequities, or ensure deep learning is occurring. Click on each risk below to view a visual representation of the problem.
The Dual Feedback Loop: AI & Teacher Synergy
The most powerful implementation of GenAI in education pairs Learning Analytics with both AI and the human teacher. This creates two interconnected feedback loops: one for immediate, automated student support, and another for strategic, teacher-led class intervention.
Student Interaction
Student asks multiple questions about separation of variables, indicating confusion.
💡 Learning Analytics Insight
Pattern detected: Student consistently struggles with recognizing when to apply separation of variables.
Loop 1: Real-Time AI Adaptation
The AI system uses the insight to immediately adjust its approach for the individual student.
🤖
GenAI Adapts
Proactively offers a visual walkthrough on identifying separable equations and the step-by-step process
Outcome: Immediate Support
Loop 2: Teacher-Informed Strategy
The aggregated insight is flagged on the teacher's dashboard, informing future lesson planning.
👩🏫
Teacher Intervenes
Sees that 30% of the class is struggling and plans to re-teach the concept the next day
Outcome: Classwide Improvement
See the Difference: An Interactive Model
The model below illustrates the flow of information with and without Learning Analytics. Start with the "GenAI Alone" model, a simple one-way street. Then, click the button to see how adding a Learning Analytics engine creates a powerful, adaptive feedback loop for continuous improvement.
LA Engine
Analyzes Interaction
Teacher Dashboard
Class-level insights
Insight
Student struggles
Student
Asks a question
Generative AI
Provides an answer
Learner Model Updated
The system knows more about the learner
Current State: GenAI Alone
This is a simple, one-way transaction. The student asks, the AI answers. We have no insight into whether the student understood, if the answer was truly helpful, or how this interaction compares to others. The system cannot learn or adapt from the experience.
AI Alignment, Made Practical
Alignment means making sure AI works toward what we actually value. In education, it includes students building understanding, confidence, and equitable outcomes. In this model, Learning Analytics (LA) provides evidence from student work, interactions, and results. Those signals help the AI adapt its next step for each learner and help teachers decide what to do next for the class. The goal is to optimize for learning, not just clicks, speed, or getting the right answer once.
Alignment in the AI Loop
- Use LA signals (e.g., misconceptions, timing, self-regulation) to shape next responses.
- Update the learner model to reflect new evidence of understanding.
- Measure transfer, not only correctness, to avoid superficial gains.
Alignment in the Teacher Loop
- Surface class‑level patterns on the dashboard to guide instruction.
- Act on inequities by comparing outcomes across groups.
- Close the loop by evaluating impact and iterating.
AI Alignment in Education: a short primer
Designing and enforcing AI alignment in education requires technical, ethical and participatory strategies working together.
Implementation Pathway: Following the Learning Analytics Cycle
To harness the benefits of Generative AI while mitigating its risks, follow the Learning Analytics cycle. This iterative process ensures continuous improvement and alignment between AI deployment and learning outcomes.
Define Learning Objectives & Success Metrics
Before deploying AI, clearly articulate what successful learning looks like. Move beyond surface metrics (engagement, completion rates) to define indicators of deep understanding. For differential equations, this might include: ability to select appropriate methods, explain conceptual reasoning, and transfer skills to novel problems.
Collect Relevant Data
Build infrastructure to capture interaction data that reveals learning processes. Track not just what students answer, but how they engage: question formulation quality, time spent reasoning vs. copying, patterns in help-seeking behavior. Ensure data collection is ethical, transparent, and serves your defined objectives.
Analyze & Generate Insights
Transform data into actionable insights. Identify patterns that reveal misalignment between AI use and learning goals. Are certain student groups struggling? Is the AI fostering dependency rather than understanding? Run small-scale pilots to test hypotheses before scaling.
Take Informed Action
Use insights to intervene at both the system and class level. Adapt the AI's behavior, modify prompts or constraints, and equip teachers with dashboards showing where students need support. Train educators to interpret LA insights and integrate them into teaching practice.
Evaluate & Refine
Measure the impact of your interventions against your original learning objectives. Did changes improve alignment? Are there unintended consequences? Use evaluation results to refine your objectives, data collection, and interventions. This closes the loop and begins the cycle again.
Essential Considerations Throughout
- • Vendor Transparency: Demand that AI vendors provide access to interaction data. "Black box" tools cannot be aligned.
- • Educator Capacity: Build data literacy skills among teachers and administrators to make the cycle effective.
- • Student Agency: Involve students in understanding how their data informs improvements to their learning experience.
Beyond Metrics: Toward Meaningful Learning
The goal isn't just to measure more—it's to measure what matters. Learning Analytics provides the evidence base to ensure that as AI becomes more sophisticated and persuasive, it remains aligned with the messy, complex, profoundly human process of learning. Without this alignment mechanism, we risk building increasingly powerful tools that optimize for the wrong things.
References
Further reading and sources that informed this analysis.
- Generative AI and Learning Analytics (JLA Editorial).
- The promise and challenges of generative AI in education.
- The impact of Generative AI (GenAI) on practices, policies and research direction in education: a case of ChatGPT and Midjourney.
- Promises and challenges of generative artificial intelligence for human learning.
- The interplay of learning, analytics and artificial intelligence in education: A vision for hybrid intelligence.
- Generative AI for Customizable Learning Experiences.
- New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution (go to page 15 for concrete Benefits and Challenges.)