Blog6 min read
How to become an AI engineer
A plain route from your first model to agents in production.
The short answer
An AI engineer builds products on top of models: prompts, retrieval, evaluation, agents, and the serving and cost work that keeps them running. You don't need to train frontier models, but you do need to know how they work. A good order is the basics of machine learning, then how transformers work, then prompting, evaluation, retrieval, agents and production. At about ten hours a week, the stages take three to four months, plus time for your projects. Show it with three projects you have measured.
What the job is
Building with models, and how it differs from ML engineering.
AI engineers build software that uses large language models and other models to do useful work: an assistant that answers from your documents, an agent that files tickets, a pipeline that pulls fields out of contracts. The model is one part, like a database. The job is everything around it.
| Row | AI engineer | ML engineer |
|---|---|---|
| Main work | Builds products on models others trained | Trains and ships models on the company's own data |
| Daily tools | Model APIs, prompts, retrieval, evals, agents | Training pipelines, features, GPUs, model monitoring |
| Success means | Good answers, fast enough, at a cost that works | Accurate models that hold up over time |
The line is blurry, and many people do some of both. AI engineer vs ML engineer sets the two side by side.
Before you start
The working basics to have in hand first.
- Python you can write without looking up every line: functions, classes, files, and a library or two such as pandas.
- Git and the command line: branches, commits, running scripts.
- Calling a web API and reading the JSON that comes back.
- School algebra and a little statistics: averages, percentages, what a distribution is.
If any of these are shaky, the Engineering Toolkit covers the shell and git, and you can pick up the rest along the way.
The route, in order
Seven stages, each building on the one before.
| Stage | What you learn | Time |
|---|---|---|
| 1. How models learn | Training, validation and test sets, overfitting, honest measurement | 2 weeks |
| 2. Neural networks and transformers | Neurons, gradients, tokens, embeddings, attention, the context window | 3 weeks |
| 3. Prompting | How a chat request is built, few-shot examples, structured output | 1 week |
| 4. Evaluation | Specifying good output, rubrics, reading failures, metrics | 2 weeks |
| 5. Retrieval | Keyword and vector search, combining them, reranking, grounded answers | 2 weeks |
| 6. Agents | The agent loop, tools, memory, limits, and when a fixed workflow is better | 2 weeks |
| 7. Production | Serving, latency, caching, routing between models, fallbacks, cost | 2 weeks |
The order matters more than the pace. Each stage uses the one before: you can't judge a retrieval system without evaluation, and you can't debug an agent without knowing what fills its context window.
Why evaluation comes early
Most AI features fail quietly: the answer looks fine and is wrong. The engineers who stand out write down what a good answer is, build a test set from real cases, and measure every change against it. Learn that early and every later stage gets easier. Specifying a good output is the place to start.
insideout's AI engineering track follows this order, from Classical Machine Learning to Production AI.
The maths you need
Enough to reason about models, without a degree's worth.
You need less maths than people fear, but not none. Be comfortable with vectors and dot products, which is how attention and vector search score things; probabilities and softmax, which is how a model picks a token; averages and spread, which is how you read eval results; and the idea of a gradient, the direction a number should move to lower a loss.
Example 4.1
Is the new prompt really better?
Your old prompt answered 78 of 100 test questions correctly. A new one answers 84. Is that a real improvement?
- The gain84% − 78% = 6 points
- Rough standard error of each score√(0.8 × 0.2 ÷ 100) = 0.04
For a share p measured on n questions, the standard error is about √(p(1 − p) ÷ n). Here p is near 0.8.
- Standard error of the difference√(0.04² + 0.04²) ≈ 0.057
Treating the two runs as independent. Comparing the prompts question by question shrinks it.
- Compare the gain with the noise0.06 ÷ 0.057 ≈ 1.05
The gain is about one standard error, well within what chance alone produces.
AnswerNot yet. Grow the test set, or compare the two prompts question by question to cut the noise.
That kind of reasoning comes up every week, and it's what Evaluation teaches.
Count the cost
Tokens in, tokens out, times the traffic.
Example 5.1
What a support assistant costs
Each request sends 2,000 tokens in and gets 300 tokens back. Say the model charges $3 for every million tokens sent in and $15 for every million it writes, and you serve 50,000 requests a day.
- Input cost per request2,000 × $3 ÷ 1,000,000 = $0.006
- Output cost per request300 × $15 ÷ 1,000,000 = $0.0045
Output tokens usually cost several times more than input tokens.
- Cost per day$0.0105 × 50,000 = $525
AnswerAbout $525 a day, roughly $16,000 a month.
Then ask where to save: a shorter prompt, a cache for repeated questions, or sending easy requests to a cheaper model. These are example prices, so check your provider's current list. Production AI covers routing, caching and budgets.
Show it with projects
Three projects that cover the whole job.
- An assistant with a test set. It answers questions from a set of documents, and you report its score on 50 real questions after every change.
- A retrieval system you measured. Compare keyword, vector and combined search on the same questions, and show the numbers.
- An agent with limits. A tool-using agent with a step budget, logged traces, and a clear way of stopping when it gets stuck.
Write up each one: what you tried, what the numbers said, what you'd do next. Interviewers read that write-up. Every insideout module has a project you build on your own machine.
The interview
What AI engineer interviews test, round by round.
AI engineer interviews usually mix a build round or take-home, a conversation about evaluation, design of a product with a model inside, questions on how models work, a coding round, and a talk about your past work. The AI engineer interview page maps each round, with practice.
- Build round. Ship something small that works, and say how you would know it works.
- Evals. You'll be asked how you would measure a feature. Have a habit, with numbers.
- System design. A product with a model inside: retrieval, caching, fallbacks, cost and latency.
- How models work. Tokens, attention, context windows and sampling. How large language models work is a quick refresher.
Learn it properly
- Training, validation and test setsClassical Machine Learning · Lesson 03 Free
- Attention as a soft lookupTransformers · Lesson 02 Free
- How a chat request is builtPrompting · Lesson 01 Free
- Specifying a good outputEvaluation · Lesson 01 Free
- Keyword search with TF-IDF and BM25Search and Retrieval · Lesson 01 Free
- Observe, decide, actAgents · Lesson 01 Free
- Routing requests between modelsProduction AI · Lesson 01 Free
Questions
Do I need a degree to become an AI engineer?
No. Employers look for evidence that you can build and measure working systems. A computer science background helps, but projects with honest results and a clear write-up count for a lot.
Do I need to learn how to train models?
You should understand how training works, so you know what fine-tuning can and can't do. Most AI engineering uses models someone else trained, adapted with prompts, retrieval and sometimes light fine-tuning.
How long does it take to become an AI engineer?
For a working developer, three to four months at about ten hours a week covers the skills, then a month or two for projects. Getting hired also depends on the projects you can show.
Should I learn Python or JavaScript?
Python first. It is the default for data, evaluation and most model tools. JavaScript or TypeScript is common for the product around the model, and easy to add later.