Since the sudden emergence of ChatGPT and other AI chatbots, many teachers and professors have started using AI detectors to check their students' work. The idea is that if a student gives the robot a task, a detector will pick it up.
However, this approach is controversial because these AI detectors have been shown to return false positives in some cases, claiming that the text was generated by AI, even when the students did all the work themselves without the help of a chatbot. False positives seem to occur more often among students whose first language is not English.
So some instructors are trying a different approach to preventing AI cheating: borrowing a page from forensics.
This is called “linguistic fingerprinting” and uses linguistic techniques to analyze previous writing to determine whether the text was written by a specific person. This technique, sometimes called “author identification,” helped catch terrorist Ted Kaczynski, known as the Unabomber, for his series of deadly mail bombs. Please help me identify him.
Mike Kentz was an early adopter of the idea of introducing this fingerprinting technology into classrooms, claiming that the approach “flips the script” on common methods of checking for plagiarism or AI. He is an English teacher at Benedict Military Academy in Savannah, Georgia, and also writes a newsletter on the issues AI raises in education.
Kentz shares his experience with the approach and discusses the pros and cons on this week's EdSurge Podcast.
Hear the full story in this week's episode. Listen on Apple Podcasts, Overcast, Spotify, or wherever you listen to podcasts, or use the player on this page. Or read the partial transcript below, lightly edited for clarity.
EdSurge: What is language fingerprinting?
Mike Kents: It's very similar to a regular fingerprint, except it has to do with the way we write. And the idea is that each of us has a unique way of communicating that allows us to create, track, and identify patterns. If you have a document known to have been written by someone, you can assign a pattern to that person's fingerprint.
How is it used in education?
If you have a document known to have been written by a student, you can run the latest essay submitted against the original passage and see if the language style matches the syntax, word choice, and vocabulary density. …
And there are tools to generate reports. And I'm not saying, 'Yes, this kid wrote it' or 'No, that student didn't write it.' It's in a spectrum, and inside the system there are a lot of vectors on a kind of pendulum. The likelihood that the author of the first paper also wrote the second paper is expressed as a percentage.
I understand that this approach has been helpful at your school recently. Can you share it?
My freshman science teacher came up to me and said, 'Hey, we have a student whose writing doesn't really sound like him.' 'Is there any other article I can compare and check so I don't end up criticizing the person when he doesn't deserve it?' And I said, 'Yes.'
And we ran through it. [linguistic fingerprint tool] And I wrote a report. The report confirmed what we thought was unlikely to have been written by the student.
The biology teacher went to the mother – she didn't even need to use the report – and said it didn't look like the student had written it. And it turns out his mom wrote some for him. So in this case it wasn't AI, but he actually didn't write it.
Some critics of this idea have pointed out that students' writing should change as they learn, so passages based on previous writing samples may no longer be accurate. Shouldn’t students’ writing change?
If you've ever taught middle school writing, like me, or early high school writing, you'll know that their writing doesn't change much after eight months. Yes, I hope it improves. Yes, it's getting better. But we're talking about very sophisticated algorithms, so even if you have a great writing teacher, it's not going to change that much in eight months. And you can always run a new assignment to get new “known documents” for their writing later in the semester.
Some people might worry that because this technology comes from law enforcement, it has some kind of criminal justice vibe.
Even if a situation arises next year where we believe a child has used AI, we will not immediately subject them to a fingerprinting process. That won't be the first thing I do. Let me talk to them first. Fortunately, there is enough trust there and we can figure it out. But I think this is just kind of a good backup, just in case.
Schools should have a system of rewards and consequences, and a system of enforcing rules and disciplining children when they step out of line. for example, [many schools] There's a camera in the hallway. What I mean is, we do it so we have documented evidence in case something goes wrong. We actually have all kinds of disciplinary measures in place, backed up by mechanisms to ensure that such things are deterred.
How optimistic are you that this and other approaches you are experimenting with will work?
I think we're going to have a very rough next five years, maybe longer. I believe that the Ministry of Education and local governments should establish AI literacy as a core competency in schools.
And we need to change our assessment strategies and what we care about what children produce, and acknowledge that written work is no longer going to be that way. You know my new thing is also verbal communication. So when he's finished his essay, I'm doing more work where I'm talking now. Everyone will go up without papers and talk about their case for 3-5 minutes. Whatever it is, your job is to verbally convey what you were trying to claim and how you were going to prove it. This is because it is something AI cannot do. So my optimism lies in rethinking our evaluation strategy.
My bigger fear is that trust in the classroom will erode.
I think there will be big problems at school next year. The student said, ‘Yes, I did that. [AI], but it's still my job.' Then the teacher says, ‘No matter how much you use, it’s too much.’
Or what is too much and what is too little?
Because any teacher can tell you that it's a delicate balance. Classroom management is a delicate balance. You are always managing your children's emotions and where they are at for the day, as well as your own. And you're trying to develop trust, maintain trust, and grow trust. We have to make sure that this very delicate, beautiful and important thing doesn't fall to the ground and break into a million pieces.
Listen to the full conversation on the EdSurge Podcast.