Cloud vs. Local LLM Agents for Generating Lecture Slides with an Agent Skill

A preliminary study. Each model ran once per task, so these are early indications, not firm conclusions. More tests are needed.

The Study, in Short

Each model got one course module and one fixed prompt: turn it into a 20–30 slide deck for a 50-minute lecture, with speaker notes on every slide and a quiz at the end. There were two subjects, Wireless 101 and STATS 101. The skill's checker and a blind LLM judge scored every finished deck.

Generated Decks: the Main Comparison

One run per model per subject. Each deck is one HTML file and opens in a new tab.

Wireless 101

• Opus 5.530 slides · judge 9/10 · 11m 44s · $2.88
• Sonnet 5.530 slides · judge 9/10 · 5m 27s · $0.90
• Haiku 5.529 slides · judge 7/10 · 7m 05s · $0.24

qwen3.8 27B, the local model, reached the 120-minute limit on Wireless 101 without writing any slides, so there is no deck.

STATS 101

• Opus 5.530 slides · judge 9/10 · 11m 14s · $2.79
• Sonnet 5.529 slides · judge 9/10 · 5m 24s · $0.70
• Haiku 5.529 slides · judge 7/10 · 8m 44s · $0.32
• Assembled afterwardsqwen3.8 27B (local)Timed out at 120 minutes. 20 slides from its unfinished run. Not judged.

Generated Decks: the Plan Experiment

A separate experiment, not compared with the runs above. Opus wrote one plan for STATS 101, and three local models built a deck from it.

• qwen3.6 35B-A3B (MoE, local)30 slides · judge 9/10 · 20 minutes
• Assembled afterwardsqwen3.8 27B (local)Stopped by an API error. 10 slides from its unfinished run. Not judged.
• Assembled afterwardsgemma4 26B (local)Stopped early. 10 slides from its unfinished run. Not judged.

The decks marked “Assembled afterwards” were not made by the models. I put them together after the runs from the slide files the unfinished local runs left, so their slides can be seen. They were not checked or judged, and they are not counted in any result.

Source, Data and Scripts

The prompts, the harness, the judge and the scripts are on GitHub: present-in-html/evals/model-comparison. The numbers behind the figures are here too: study-data.json and tables.md.