Prompy
Hamel HusainArticle

ArticleIntermediate

Your AI product needs evals

Hamel Husain · hamel.dev

Hamel Husain's case study of how a real product team got unstuck: unit tests for prompts, looking at your data every day, human and model review, and why a cheap evaluation loop is what lets you change prompts with confidence.

Keep going

All resources
AnthropicDocs

DocsIntermediate

Define success criteria and build evaluations

How to decide what "good" means before you tune a prompt, then test for it: specific, measurable success criteria, building test cases that include edge cases, and grading by code, by a model or by a person. The step most people skip.

Anthropic#evals#claude

VideoIntermediate3 h 31 min

Deep dive into LLMs like ChatGPT

A three-and-a-half-hour walk through the whole stack: tokenisation, pretraining, fine-tuning into an assistant, hallucinations, tool use and reasoning models. Long, but it explains why the prompting advice in the other guides works.

Andrej Karpathy#llm-basics
Simon WillisonArticle

ArticleIntermediate

Prompt injection: the series

Simon Willison coined the term and has tracked it since 2022. Read the earliest posts for the core problem (a model cannot reliably tell instructions from data), then the recent ones on what it means for agents with tools. Required reading before you connect a model to anything private.

Simon Willison#security