These days, new educational technology products are hitting the market, and teachers and professors are creating more and more educational videos and other materials for their classes. However, one group that is often left out of the design process is students.
“Many educational products aren’t shown to students until they’re already designed,” Elliott Hedman, a consultant who works for educational technology companies, said in a talk at this month’s SXSW EDU festival.
Although most major media and consumer technology products undergo extensive testing these days, many adults have pointed out that they are okay with holding much lower standards for any product designed for children.
“The budget for one episode of Game of Thrones was $15 million. It was probably worth it. “It was a very good episode,” he said. “I am sometimes hired to write reading novels for children in classrooms and am paid $80. “This is what we are willing to pay for our children’s classroom experience.”
Hedman acknowledges that this is probably an extreme example, but he says the lack of testing really lets students know when material is being presented to them. When he visited school computer labs for a research project, he says, “I noticed that a lot of kids would alt-tab out of their educational programs and start watching YouTube for an hour.”
In an interview following the presentation, Hedman, who holds a Ph.D., said: He, who is a graduate of MIT Media Lab and has been working on educational material design for over 10 years, said that it does not mean that edtech companies do not test at all. But he said the tests they do are often ineffective or too limited, relying primarily on families of company employees who are “well above average in school and very white.”
He points out that there is no good incentive for ed-tech companies to spend time and effort on more detailed testing of students. “They are selling to governments, administrations and regions,” he points out. “They are not selling to children. Children have no purchasing power. The children can't hear at all and the teachers can barely hear. Then we throw it in the classroom and they test it, ‘Did your score go up?’”
And he argues that conducting more user testing with students during the design process is different from conducting efficacy studies of an educational technology product. This has been a demand from a growing number of education leaders in recent years.
“I’m not saying efficacy research is a bad thing,” he says. But most research on whether teaching methods are effective “can be a hindrance” to conducting design research, he says. A change that can correct all failures.
He said the ideal approach would be to place an edtech product or learning material in front of a diverse group of students without a researcher in the room and videotape how they use the tool. Designers should then make small improvements based on what they've learned and continue to do so in an iterative manner throughout the development process, he says. That way, it's more of a co-creation with students rather than adults creating something and assuming it will work for the kids.
Experts also called for more teachers and educators to participate in the development of edtech products.
Instructor as Designer
Increasingly, instructors themselves are the creators of learning materials. For example, if you're trying a “flipped classroom” approach where students learn some basic material at home, allowing class time to be used for more interactive activities, such as short video lessons. . And some experts say that too often students are left out of the design process for these materials, too.
“The challenge with using materials that don’t have a good user experience is that they make learning more difficult,” says Kayla B. McNabb, associate director of teaching and learning engagement at Virginia Tech. She said, “If we want students to do well in our courses, we need to lower the barriers to engaging in learning experiences.”
McNabb, who earned his Ph.D. in Rhetoric and Writing from Virginia Tech, he co-authored a paper in 2021 urging educators to do more to incorporate user testing when designing learning materials.
In many cases, what instructors learn by running short video samples that a few students have created for class isn't about the content, but about the user experience, McNabb told EdSurge in an interview. And the problems raised by students are often easy to solve and apply to other projects.
“The transition to video can be too jarring,” she says. “Then let’s add a different header image to make it more clear.”
McNabb says one of the major challenges for universities is that many instructors have never been trained in how to teach, let alone how to design learning materials. But even just taking a little time to share some of the course material with students to get their input can help instructors make changes that significantly improve their impact, she says.
“The most important thing people can do is ask for feedback,” she says, noting that it can be effective at any time during the development process. “Anytime is better than never.”
If an instructor cannot find students to test the material, suggest testing it on a colleague or presenting it at a conference. “Any feedback is better than no feedback,” she argues.
changing times
Hedman says the need for product testing may become more important as companies rush to add new AI features to their products.
“The way children use AI tools is very different from what we think,” he says. “I want to remind companies. You have never shown anything like that to a child. “This will become critical as companies rush to launch AI-based products.”
But he says doing so would change the culture of many companies.
“In an ideal world, I would like the student voice to be front and center of every edtech company,” he says.
User design researcher Yu-Chen Chiu argued in a recent article on Medium that one way to solve this problem is to increase collaboration between university researchers and the edtech industry. The goal, she claims, is “to develop impactful products that are not only ‘effective,’ but also ‘usable’ and ‘desirable.’”
Now that students have more ways than ever to find information online or use AI chatbots, their engagement is also important. This means that instructors should focus more on teaching skills and ways of thinking in their field rather than spending too much time on specific content or details that can change quickly or that students can find elsewhere.
Essentially, she argues that the rise of AI means many instructors will need to update materials they created in the past, and she hopes more educators will involve their students in this redesign process.
“That doesn’t mean you have to drop everything and get a master’s degree in instructional design,” says McNabb. “This means you need to think clearly about the user experience throughout the process.”