Texas students taking state-mandated tests this week are being used as guinea pigs for a new artificial intelligence-based grading system that will replace most of the district's human graders.
Texas Tribune An “automated grading engine” that leverages natural language processing, a technology that allows chatbots like OpenAI's ChatGPT to understand and communicate with users, is reportedly being rolled out by the Texas Education Agency (TEA) to evaluate open-ended questions about the U.S. state. do. Texas Assessment of Academic Readiness (STAAR) test. The agency expects the system to save $15 million to $20 million annually by reducing the need for temporary human graders, and plans to hire fewer than 2,000 graders this year, compared to the 6,000 needed in 2023.
“We wanted to keep as many open-ended responses as possible, but scoring them takes an enormous amount of time.”
The STAAR exam, which tests students' understanding of the core curriculum in grades 3 through 8, was redesigned last year to include fewer multiple-choice questions. It now includes up to seven times more open-ended questions, and Jose Rios, TEA's director of student assessment, said the organization wanted to “keep as many open-ended responses as possible, but grading them takes an enormous amount of time.” ”
According to a slideshow hosted on the TEA website, the new scoring system has already been trained using 3,000 test responses that have been human-scored twice. Some safety nets have also been implemented. For example, a quarter of all computer-graded results are re-scored by humans, as are answers that confuse the AI system (including slang or non-English responses).
Although TEA is optimistic that AI will save them a lot of cash, some educators are not so keen on seeing AI implemented. Lewisville Independent School District Superintendent Lori Rapp said that when her automated grading system was in limited use in December 2023, she saw a “dramatic increase” in composed responses receiving a score of zero in her district. The question is whether she has a problem with the test questions or the new automated grading system,” Rapp said.
AI essay grading engines are not new. 2019 report motherboard TEA appears determined to avoid the same reputation, but has found itself in at least 21 states with varying degrees of success. The fine print in TEA's slideshow also emphasizes that the new scoring engine is inherently a closed system, different from AI. In other words, “AI is a computer that uses progressive learning algorithms to adapt and allow data to do the programming and essentially teach itself.”
Attempts to draw a line between them are not surprising. There is no shortage of teachers despairing online about how generative AI services are being used to cheat assignments and homework. Students grading with this new grading system may have a hard time accepting the way they believe “rules for you, not rules for me” applies here.