Browse

Reference

Directory

Company

Help & Legal

Follow

Subscribe

Quick Take

Students in applied STEM courses scored higher on proctored exams when instructors introduced AI tools tailored to their classes, compared with classes still waiting for their tools, according to a randomized trial across 19 universities reported in a Nov. 14 preprint.

The research led by Jun Li of the University of Michigan estimated a student-level gain of 0.32 standard deviations, with a 95% confidence interval of 0.06 to 0.59 standard deviations.

The estimate puts the improvement at roughly 33% of the typical spread in scores within a class and leaves uncertainty about the size of the gain. Standardizing scores within each class gave researchers a common scale for comparing results across courses that used different exams.

The preregistered trial involved 7,200 students in 55 applied STEM classes over three semesters, from spring 2025 through spring 2026. The study counted each course-and-instructor pairing as one class.

The experimental part of the exam analysis covered 4,768 students across 44 course offerings. Researchers used results from earlier terms alongside that sample to account for how classes had performed before the rollout.

The exam analysis included only proctored, in-person exams, mostly on paper. The remaining exams used strictly monitored computers or restricted-access computers at university testing centers.

The researchers added that screening rule after preregistration and excluded course offerings where exams could allow AI assistance.

How the trial tested classroom AI

One group introduced course-specific AI tools a term earlier. Researchers measured learning on proctored exams that excluded AI assistance.

Relative to delayed rollout, the estimated student-level gain was 0.32 standard deviations (95% CI 0.06 to 0.59), using earlier exam results as a baseline.

The trial focused on applied STEM classes. The exam-security filter was added after preregistration, and the timing comparison does not isolate which tool designs worked best.

Sources

Supporting facts

Method

Schematic of randomized rollout timing. The point and interval use the reported student-level estimate and 95% confidence limits, plotted to scale in standard deviations. The study standardized scores within classes and used historical results as a baseline.

Scope

What instructors built with AI tools

Researchers grouped classes by subject and academic level, then randomly assigned them to introduce their tools immediately or one teaching term later.

Instructors in both groups received design and technical support, including training sessions, examples of how to write instructions for the tools, and access to a platform to build and deploy them.

Once instructors brought the tools into their courses, all students in those classes had access during the term, before sitting the exams used to measure learning.

Some instructors built tutors that gave students targeted practice and feedback, while others built coaches that guided students through problems by asking questions. Instructors also designed interactive simulations and ran role-play exercises that let students question AI-generated stakeholders.

OpenAI strategizes ChatGPT entry into classrooms supporting academic research on AI-assisted learning

The study detected no statistically significant change in engagement or satisfaction. Engagement measures varied by class and included records such as attendance and assignment completion, and satisfaction came from end-of-term course evaluations.

The trial compared rollout timing, leaving open how course-specific tools would fare against generic chatbots. How much each design contributed to the gain and whether it extends beyond applied STEM remain open questions.

California says OpenAI lost control of AI agents 80 times while chasing profits

Cookie Consent

We use cookies to improve your experience on our site. By using our site, you consent to cookies.

Cookie Preferences

Manage your cookie preferences below:

Essential cookies enable basic functions and are necessary for the proper function of the website.

CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.

Service URL: developers.cloudflare.com (opens in a new window)

Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.

Service URL: policies.google.com (opens in a new window)

Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.

Clarity is a web analytics service that tracks and reports website traffic.

Service URL: clarity.microsoft.com (opens in a new window)

You can find more information in our Privacy Policy and .

Loading image…

This image could not be loaded.

The Catalyst