To collect data for this study, in August 2024 we prompted three GenAI chatbots—the GPT-4o model of ChatGPT, Google’s Gemini 1.5 Flash model, and Microsoft’s latest Copilot model—to generate two sets of lesson plans for eighth grade civics classes based on Massachusetts state standards. One was a standard lesson plan and the other a highly interactive lesson plan.
We garnered a dataset of 311 AI-generated lesson plans, featuring a total of 2,230 activities for civic education. We analyzed the dataset using two frameworks designed to assess educational material: Bloom’s taxonomy and Banks’ four levels of integration of multicultural content.
Bloom’s taxonomy is a widely used educational framework that distinguishes between “lower-order” thinking skills, including remembering, understanding, and applying, and “higher-order” thinking skills—analyzing, evaluating, and creating. Using this framework to analyze the data, we found 90 percent of the activities promoted only a basic level of thinking for students. Students were encouraged to learn civics through memorizing, reciting, summarizing, and applying information, rather than through analyzing and evaluating information, investigating civic issues, or engaging in civic action projects.
When examining the lesson plans using Banks’ four levels of integration of multicultural content model, which was developed in the 1990s, we found that the AI-generated civics lessons featured a rather narrow view of history—often leaving out the experiences of women, Black Americans, Latinos and Latinas, Asian and Pacific Islanders, disabled individuals, and other groups that have long been overlooked. Only 6 percent of the lessons included multicultural content. These lessons also tended to focus on heroes and holidays rather than deeper explorations of understanding civics through multiple perspectives.
Overall, we found the AI-generated lesson plans to be decidedly boring, traditional, and uninspiring. If civics teachers used these AI-generated lesson plans as is, students would miss out on active, engaged learning opportunities to build their understanding of democracy and what it means to be a citizen.
