Skip to main content

North Hills Monthly

Using Artificial Intelligence, Lumilo Glasses Could Help Highlight Student Comprehension in Future Classrooms

Jul 30, 2020 08:37PM ● By Erica Cebzanov

Imagine glasses that allow educators to view emoji-like indicators displaying question marks, warning signs or smiley faces in real time above their students’ heads, indicating pupils’ comprehension of classroom assignments. The technology isn’t that far away.

Three years ago, math teachers from 14 Pittsburgh-area school districts assisted in the co-design of augmented reality glasses to enhance their abilities to teach students while using artificially intelligent (AI) math software. The software could detect when a student seemed likely to need help from a human teacher, for example, even when the student was not asking for help.

A Carnegie Mellon University team invented the glasses, named Lumilo after the meaning of “luminary” and “light” in the Esperanto language that Polish ophthalmologist L.L. Zamenhof devised in 1887 as a universal world language to foster understanding.

Dr. Ken Holstein, Carnegie Mellon University Human-Computer Interaction Institute assistant professor, developed Lumilo’s app with a team including Gena Hong, Mera Tegene, Zac Yu, Vincent Aleven and Bruce McLaren. Lumilo utilizes artificial intelligence to provide students with prompts as they work on their assignments.

As part of his Lumilo research, Holstein found it important to engage with middle school teachers who had already utilized AI within their classrooms. When he was a new doctoral student, he was disheartened by the teachers’ fears that AI would “automate away” their jobs, or force new technologies upon them that were not well-aligned to their teaching.

Still, he said, “These teachers really appreciate the use of AI, even if they have a lot of misgivings about the particular ways in which they're designed, because it allows them to personalize instruction much more than they would be able to otherwise.” He likened it to having a teaching assistant for each student, even if much less capable than a human assistant.

Because Lumilo’s mission is to enhance teachers’ abilities rather than replace them, Holstein held an iterative series of design workshops to explore teachers’ wants and concerns involving the software. He says his team encourages educators to utilize context and not blindly follow the symbols that the device displays.

“They're not mind readers; machine learning and AI systems are always error-prone,” he said, giving the example of when a student appeared to complete math exercises too quickly, according to the software. The teacher discovered that the middle-schooler had broken up with his significant other over the weekend and was having difficulty focusing on math. The researcher considers this as an example of AI’s inability to replicate the teacher-student relationship.

During early design sessions, students found it violating if teachers could view their attention and emotional levels during lessons. These features were excluded from Lumilo’s prototypes used in classrooms, in favor of features that teachers and students found mutually beneficial.

During trials, teachers and students responded well to the app identifying when students were in negative spirals and having difficulty with problems. “The AI system is saying that you're, you know, kind of stuck in the mud here,” said Holstein. “You're spinning your wheels; it seems like you might need help based on your recent performance over the past five minutes.”

Since these initial trials, Holstein, McLaren and Aleven have partnered with Carnegie Learning Inc., an educational technology company delivering AI-based tutoring solutions for kindergarten- through 12th-grade learners. Together, they were planning on working beyond the Lumilo research prototype with a larger pilot in Pittsburgh and Florida classrooms until the COVID-19 outbreak postponed their plans.

The next phase will involve a more lightweight design, a greater diversity of features that provide students with greater agency over their own learning, and a longer testing period.

In the meantime, the team is working to help teachers address the challenges they are facing while teaching remotely during the pandemic. For example, Carnegie Learning’s LiveLab tool provides teachers with indicators on an electronic tablet, identifying students’ progress or need for support while they work in the corresponding MATHia lab. Though the tool was originally designed for classroom use, there are opportunities to make it more useful to teachers during remote learning.