Socrates Against The Machine: Can Looking Back Help Us to Think About the Future of Education?

In the past two weeks, AI writing has gone mainstream. OpenAI’s ChatGPT, an app sitting on top of its latest large language model, was released for “public testing”, and it seems to have captured the imagination of a large chunk of the internet.

The website passed 1 million users in the first five days. For comparison, Instagram took about 2 and a half months to reach that many users, and Netflix over three years. Whatever your opinion on the quality of the writing – and that ranges from “it’s a wooden, clunky, bullshit machine” to “this is going to spell the end of human writers everywhere!” – ChatGPT and similar models are bound to disrupt the education system. As educators, we need to learn the capabilities and limitations of the technology.

ChatGPT: The Cheating Machine

Most articles on ChatGPT and education are focused squarely on the biggest perceived threat: students using it to cheat on essays. It’s a valid fear. With a little bit of tinkering and almost no content knowledge at all, it’s easy to craft an essay that seems plausible and well written.

To test the capabilities of the Great Cheating Machine, I asked ChatGPT to dream up 5 possible areas of study which might be considered esoteric or a little left-of-centre. Here’s what it came up with:

I can confidently say that with the exception of quantum mechanics (I am a science fiction nerd after all), I have no subject matter knowledge in any of those areas. I’m going to run with number three. Imagine a course on the neuroscience of hallucinations. The content is delivered over a semester, and is assessed primarily through a well researched essay which includes a literature review and a reflection on the student’s own experiences of hallucinations (stay with me…)

After Googling a couple of choice readings for my “literature review” and dumping them into ChatGPT, here’s the start of my assignment:

Are the references accurate? I didn’t bother checking – I’m no expert on neuroscience or hallucinations, but this proves my point: ChatGPT can write. What about the second part of the task – the personal reflection? Can the GPT shift voice easily from the academic to the reflective, while maintaining its grip on the content knowledge?

Let’s see…

Well… That works.

Obviously this would need some work to meet the requirements for a complete tertiary level assignment, but now we’re into the realms of prompt engineering. Give me half an hour, and I could have this expanded out to a few thousand words.

I still don’t know anything about the neuroscience of hallucinations.

The Future of Writing in Education

So, if I can churn out a half decent written assignment in a matter of minutes, what does that mean for the future of essay writing in education? Long before ChatGPT was released I wrote a post with a few possible scenarios. Unfortunately, the most likely is that the system rejects AI writing entirely and teachers, lecturers and tutors are forced to find ways to police the technology. Theoretically there are many ways to do this. Banning and blocking the websites, writing policies forbidding the use of AI, and even cryptographically watermarking AI writing have all been tried. In practice, people will always find a way around these safeguards.

The future of writing as an assessment tool looks bleak. If we can’t effectively police the technology, then the next obvious step will be to monitor and control student behaviour. This means an increase in timed and invigilated examinations, writing by hand, and lengthy authentication processes which typically fall to the teacher to oversee.

Deakin University’s Phill Dawson has written extensively on the impact of increased authentication pressures in tertiary as a result of more students working remotely. Whether online or face-to-face, trying to crack down on cheating via surveillance places a burden on instructors and can result in uproar from students.

Fortunately, there are alternatives to this heavy handed approach, and assessment doesn’t have to be conducted via writing at all.

Join the mailing list for more articles like this

Enter Socrates

Writing serves other purposes than assessment. People write for expression, reflection, to pin down and give shape to their knowledge and understanding. If we shift the paradigm from writing being the primary tool for assessment, then we can start exploring alternatives.

There are many ways to assess knowledge beyond a written assignment. Project-based and scenario-based learning, industry and workplace observations, and oral examinations like a viva voce offer students alternative ways to demonstrate their understanding and application of a subject. I hope we’ll see an increase in methods like these in both secondary and tertiary education as the system catches up with the realities of AI writing technologies.

Amongst these practices is the age-old Socratic method; a dialogue between student and teacher designed to prompt the student through increasingly thought-provoking questions. Both student and teacher may ask questions, with the process designed to probe any underlying assumptions, knowledge gaps, or biases in understanding. In the Socratic method there is a tacit acknowledgement that neither the teacher nor the student has all the answers: the questioning should not be adversarial, but tentative and even playful.

Questioning of this kind quickly demonstrates the depth and extent of a student’s knowledge. Even if the student had used an AI to write an initial response, it would quickly become apparent that the knowledge was built on shaky foundations – not dissimilar to a student who Googles the answers but never goes beyond the first couple of links in the search results.

The Socratic Seminar

One criticism of the Socratic method is that it can be too intimidating for some students. Faced with direct questioning and lacking in thinking time, students may find themselves unable to answer. There is also a time factor. Conducting a rich and lengthy dialogue with every student in a class may be possible if you have 15-20 students, but in a larger secondary class, or a tertiary course with a ratio of hundreds of students to a single lecturer, the method is impractical.

Socratic seminars take some of the ideas from the Socratic method and turn them into an effective group activity. This is the basic process:

1. Choose a text or topic that invites authentic inquiry
2. Prepare students by letting them know when a Socratic seminar will be held and providing them with tools for annotating the text or taking notes on the topic
3. Prepare open-ended questions for the seminar
4. Establish student expectations for behaviour during the seminar
5. Establish the teacher’s role as facilitator rather than participant
6. Assess the effectiveness of the seminar through reflection and goal-setting.

Read Write Think has a straightforward description of the full process here.

Beyond the Cheating Machine

This is where we can bring ChatGPT and other LLMs into the equation. Artificial Intelligence technologies should – ideally – reduce human labour in tasks which can easily be automated. At their best, they can also provide inspiration and creative ideas.

Imagine a Socratic seminar discussing this overarching topic: How do monstrous or inhuman characters in gothic literature reflect the fears and anxieties of their respective time periods, and what do they reveal about the nature of humanity? Here are a few ways in which we might use ChatGPT to support the seminar:

Students use ChatGPT to build their knowledge

Students might use ChatGPT as a research tool. GPT 3.5, the current model, has issues with reliability. In the near future GPT 4 and Google’s LaMDA will likely overcome some of these issues. Students will still need to review the materials with a critical eye. In a sense, students are using the LLM as a “super-Google”, reducing the time spent trawling multiple websites for information.

Note the development here from broad question about the topic and a request for further reading, followed by some more specific questions. Also, check out how ChatGPT’s limited “memory” function is keeping the answers filtered down around the initial topic of inhumanity and fear:

Using ChatGPT to support the teacher

Step 3 above requires the teacher to provide questions for the seminar. Providing the questions to students ahead of time allows them to order their thoughts and prepare responses, and makes for a richer discussion. Using a resource from Facing History.org I provided ChatGPT some generic sentence stems for suitable questions, and asked it to provide some questions tailored for our discussion:

Some of these would need tweaking. Question 4, for example, is more like a probing question that might occur mid way through a discussion. Nonetheless, this would save the teacher a lot of preparation time.

Moves and countermoves

Part of the Socratic method is debate and discourse. Whether or not you included this kind of back-and-forth in the Socratic seminar, it would be a useful exercise to teach students how to defend their positions. Because ChatGPT is trained on dialogue, it is well suited to “role playing” and can be used in a number of scenarios to help refine and test arguments. Here are a few examples of ways in which students (or teachers) could use the bot to work on their debating skills:

Classrooms filled with robots

One of the biggest risks to education is not keeping pace with these technologies. Whether we like it or not, students will use AI writing tools to cheat. On the flip side, there will be many companies using the technologies to produce tools and apps which “assist” teachers with marking and feedback. I’ve already tried using ChatGPT to critique some of my fiction writing, with pretty good results.

If we’re not careful, we might end up in a situation where we have essays written by AI, marked by apps underpinned by AI. In the process, no one learns anything at all.

AI isn’t going to replace human writing any time soon. But it does mean that we need to rethink how and why we teach and assess writing. We can adopt these tools and work with students on their ethical, appropriate, and creative use. We can use AI tools as another opportunity to build strong relationships through dialogue and discourse with our students. If we don’t, we might as well replace ourselves with robots.