Builder

LLMOps / AI Platform

Keep models cheap, observable, and gated. Production is the product.

2–3 weeks · traces, cost, eval in CI

0/13 lessons on this path

Mastery syllabus

1. 1 · Quality you can fail the build on

   Groundedness in CI.

   • Measure before you brag11 min
   • Proof over confidence11 min
   • Design the AI feature10 min
   • A gold thread through the miss12 min
   • Tokens are rent12 min

2. 2 · Cost, tools, rollback

   Caps and a kill switch.

   • Tools, not magic10 min
   • Where AI must not decide9 min
   • Hours you can show10 min

3. 3 · CI and on-call

   Red builds are kindness. Five lines at 2am.

   • A red build is a kindness12 min
   • On-call is five lines, not theatre12 min
   • The user is not the boss of the rule12 min

4. 4 · JSON and rollback

   Gates read forms. Undo the last prompt.

   • Ask for a form, not a poem12 min
   • Five lines after a miss12 min

Simulation labs

• Gate in CI

  A drop in groundedness fails the build.

  Open the lab

• Tool permissions

  Search cheap. Pay waits.

  Open the lab

• Red CI

  A drop fails the build.

  Open the lab

• On-call card

  Ten lines, named page.

  Open the lab

The project that hires

An eval gate that can fail CI

1. Five gold cases in the repo.
2. A score threshold that blocks merge.
3. A one-page runbook for cost spikes.

Interview room

• How do you catch a silent quality drop after a prompt change?
• Cost doubled overnight. Walk the first four checks.
• Design rollback for a bad system prompt.
• Injection in a retrieved PDF. First move?

Resume lines that hire

• Put groundedness evals in CI and cut ungrounded answers before release.