Notes from building study tools with LLMs

Study tools look simple from the outside: upload a document, get cards back. Underneath, that flow is a pipeline — parsing, chunking, generation, review, and storage — and each stage can fail independently.

The first version I shipped generated everything synchronously. It worked for short notes and broke for longer PDFs. Moving heavier work behind a queue forced clearer boundaries: what the user waits for, what can finish later, and how the UI shows stale versus ready content.

Prompt quality still matters, but evaluation mattered more. I started saving a small set of real study materials and checking whether the cards were specific, answerable, and faithful to the source. That beat endless prompt rewriting.

If you are building learning products with LLMs, invest early in the pipeline and in a tiny evaluation set. The model is rarely the only bottleneck.