Twenty questions — two from each lesson — in four sections: Foundations (1–4), Test Data & Scoring (5–8), Guardrails (9–10), LangSmith & LangGraph (11–14), Copilot Studio (15–16), Production & Maintenance (17–20). Answer all of them to see your grade.
🧠 Grand Quiz
Foundations
1. Why does the "Software 2.0" framing make your eval dataset the source of truth for your app?
2. What is the difference between a playground check and an eval?
3. Which benchmark measures resolving real GitHub issues end-to-end?
4. Why record the full comparison matrix (score, latency, cost, structured-output failures) instead of just score?
Test Data & Scoring
5. What is a good day-one target size for a stratified golden dataset?
6. Why must eval datasets be versioned?
7. Which evaluator type is best for scoring open-ended answer quality against written criteria?
8. What is the name of the documented LLM-judge bias where longer, fancier answers score higher even when worse?
Guardrails
9. Which OWASP LLM risk is about unvalidated model output flowing into SQL, HTML, or shell?
10. What is the primary defense against LLM07 System Prompt Leakage?
LangSmith & LangGraph
11. Which LangSmith feature is the human-review loop for traced production runs?
12. In LangSmith, what makes a regression test a regression test?
13. Which LangGraph capability lets you replay a real production thread against a new graph version?
14. What should agent eval examples record beyond the expected final answer?
Copilot Studio
15. What does "grounding" mean in Copilot Studio?
16. What gives Copilot Studio a staging gate analogous to CI?
Production & Maintenance
17. What is the correct response when a production gate fails?
18. What does a canary rollout do?
19. Which of the following is NOT one of the four drift types to monitor?
20. What does the dataset refresh rhythm look like once you're in production?