insideout Get Pro
ML Engineer

ML Engineer

The night before

Read it tonight, then once more on the way in.

Back to the rounds

1The day

  1. CodingAlgorithm problems plus ML code, like a small training loop or a metric written from scratch.
  2. ML basicsQuick questions on how models learn and fail: overfitting, choosing a loss and a metric, and why accuracy alone misleads.
  3. ML system designDesign an ML system end to end: the data, the model, how it's served, and how you'd notice it getting worse.
  4. DebuggingFind why a model, a search step or a test is wrong, sometimes without AI tools.
  5. Your projectWalk through something you built: the trade-offs, what failed, and what you'd change.

2How to answer

Design an ML system end to end45 min

  1. The problem and the target6 min
  2. Sketch the pipeline4 min
  3. Data and labels10 min
  4. Model and loss10 min
  5. Offline and online evaluation8 min
  6. Serving, monitoring and drift7 min

Tell a story from your work2 min

  1. The situation in one line15 sec
  2. What you did60 sec
  3. The result, with a number30 sec
  4. What you'd change15 sec

Walk through your project10 min

  1. The problem2 min
  2. Your choice and its alternative3 min
  3. What broke2 min
  4. The result2 min
  5. What's next1 min

3Numbers to have ready

Say each answer out loud, then open the fold.

  • One payment in 1,000 is fraud. The accuracy of a model that always says no?

    99.9%, with a recall of zero. Measuring a classifier

  • 40 flagged, 30 of them right, 50 real positives. Precision, recall and F1.

    0.75, 0.60, and F1 = 2 × 0.75 × 0.60 / 1.35, about 0.67. Measuring a classifier

  • A false alarm costs 1 and a miss costs 9. Where do you flag, on calibrated scores?

    When p is above 1 / (1 + 9) = 0.1. Measuring a classifier

  • The first loss of a language model with a 32,000-token vocabulary that knows nothing.

    ln 32,000, about 10.4. A first loss far from it means a bug. Pretraining

  • Four times the test items does what to the wobble of a score?

    Halves it. It shrinks with √n. Is the new version really better

4Traps

  • The model is 99.9 percent accurate.When positives are rare, always guessing the common class scores that too. Give precision and recall at the cut you'd use, or the money.
  • We decided, we built it, and we fixed it.They're hiring you. With "we" all the way through, they can't tell what you did.
  • The stack was Python, Postgres, Airflow and Docker.A list of tools isn't a decision. Say what you chose, what you turned down, and why.
  • I'd shuffle the data and hold out 20 percent.Rows close in time share the same world, so a shuffled test leaks the future. Train on earlier data and test on later.

5On the day

  • Ask which AI tools, if any, you may use in each round.
  • Say your plan before you solve, so they can steer you early.
  • Stuck for a minute? Say what you've tried and ask for a hint. Taking one well counts.
  • Check units and sizes out loud before you give a number.
  • Name the baseline you'd beat before you name a model.