Evaluation & AI Reliability
Fictional sample profileJordan teaches how to know whether an LLM feature is good enough to ship — and how to notice when it stops being good enough.
How Jordan reviews work
Jordan reviews evaluation work by asking what would have to be true for a metric to be misleading. Feedback tends to focus on test coverage, grader reliability and how thresholds were chosen.
01
Jordan's teaching centers on evaluation as an everyday engineering habit rather than a one-off exercise. Learners write test cases before they tune prompts, define what a good output looks like and build harnesses that make comparisons repeatable.
The modules Jordan leads cover deterministic checks, rubric-based grading, how to validate an automated grader against human judgement and how to run regression tests in CI. Jordan is careful to discuss the limits of each method, including where automated grading can mislead.
Jordan believes a small, well-chosen test set that runs on every change is worth more than a large one that no one runs.
This is demo content. It does not describe a real person, employer, qualification or track record.
02
03
EvaluationChecklist8 min read
A repeatable way to decide whether your LLM feature is getting better or worse: define quality, build a representative eval set, layer cheap and expensive checks, and read results honestly.
By Jordan Lee · sample author
EvaluationGuide11 min read
A one-line prompt edit or a model upgrade can quietly break behavior that used to work. This guide shows how to build a golden set, combine deterministic and model-graded checks, and run them in CI so changes ship with evidence.
By Jordan Lee · sample author
Build the foundations, projects, and habits needed to design more dependable AI applications.