# How to become an AI engineer: what to learn, in order

A plain route from your first model to agents in production.

URL: https://insideout.sh/blog/ai-engineer-roadmap
Published: 2026-11-11
Updated: 2026-11-12

An AI engineer builds products on top of models: prompts, retrieval, evaluation, agents, and the serving and cost work that keeps them running. You don't need to train frontier models, but you do need to know how they work. A good order is the basics of machine learning, then how transformers work, then prompting, evaluation, retrieval, agents and production. At about ten hours a week, the stages take three to four months, plus time for your projects. Show it with three projects you have measured.

## 1. What the job is

AI engineers build software that uses large language models and other models to do useful work: an assistant that answers from your documents, an agent that files tickets, a pipeline that pulls fields out of contracts. The model is one part, like a database. The job is everything around it.

|  | AI engineer | ML engineer |
| --- | --- | --- |
| Main work | Builds products on models others trained | Trains and ships models on the company's own data |
| Daily tools | Model APIs, prompts, retrieval, evals, agents | Training pipelines, features, GPUs, model monitoring |
| Success means | Good answers, fast enough, at a cost that works | Accurate models that hold up over time |

The line is blurry, and many people do some of both. [AI engineer vs ML engineer](https://insideout.sh/compare/ai-engineer-vs-ml-engineer) sets the two side by side.

## 2. Before you start

- **Python** you can write without looking up every line: functions, classes, files, and a library or two such as pandas.
- **Git and the command line**: branches, commits, running scripts.
- **Calling a web API** and reading the JSON that comes back.
- **School algebra and a little statistics**: averages, percentages, what a distribution is.

If any of these are shaky, the [Engineering Toolkit](https://insideout.sh/courses/engineering-toolkit) covers the shell and git, and you can pick up the rest along the way.

## 3. The route, in order

| Stage | What you learn | Time |
| --- | --- | --- |
| 1. How models learn | Training, validation and test sets, overfitting, honest measurement | 2 weeks |
| 2. Neural networks and transformers | Neurons, gradients, tokens, embeddings, attention, the context window | 3 weeks |
| 3. Prompting | How a chat request is built, few-shot examples, structured output | 1 week |
| 4. Evaluation | Specifying good output, rubrics, reading failures, metrics | 2 weeks |
| 5. Retrieval | Keyword and vector search, combining them, reranking, grounded answers | 2 weeks |
| 6. Agents | The agent loop, tools, memory, limits, and when a fixed workflow is better | 2 weeks |
| 7. Production | Serving, latency, caching, routing between models, fallbacks, cost | 2 weeks |

Times assume about ten hours a week.

The order matters more than the pace. Each stage uses the one before: you can't judge a retrieval system without evaluation, and you can't debug an agent without knowing what fills its context window.

### Why evaluation comes early

Most AI features fail quietly: the answer looks fine and is wrong. The engineers who stand out write down what a good answer is, build a test set from real cases, and measure every change against it. Learn that early and every later stage gets easier. [Specifying a good output](https://insideout.sh/courses/evals/01) is the place to start.

insideout's [AI engineering track](https://insideout.sh/tracks) follows this order, from [Classical Machine Learning](https://insideout.sh/courses/classical-ml) to [Production AI](https://insideout.sh/courses/production-ai).

## 4. The maths you need

You need less maths than people fear, but not none. Be comfortable with vectors and dot products, which is how attention and vector search score things; probabilities and softmax, which is how a model picks a token; averages and spread, which is how you read eval results; and the idea of a gradient, the direction a number should move to lower a loss.

**Example 4.1: Is the new prompt really better?**
Problem: Your old prompt answered 78 of 100 test questions correctly. A new one answers 84. Is that a real improvement?
1. The gain: 84% − 78% = 6 points
2. Rough standard error of each score: √(0.8 × 0.2 ÷ 100) = 0.04 (For a share p measured on n questions, the standard error is about √(p(1 − p) ÷ n). Here p is near 0.8.)
3. Standard error of the difference: √(0.04² + 0.04²) ≈ 0.057 (Treating the two runs as independent. Comparing the prompts question by question shrinks it.)
4. Compare the gain with the noise: 0.06 ÷ 0.057 ≈ 1.05 (The gain is about one standard error, well within what chance alone produces.)
Answer: Not yet. Grow the test set, or compare the two prompts question by question to cut the noise.

That kind of reasoning comes up every week, and it's what [Evaluation](https://insideout.sh/courses/evals) teaches.

## 5. Count the cost

**Example 5.1: What a support assistant costs**
Problem: Each request sends 2,000 tokens in and gets 300 tokens back. Say the model charges $3 for every million tokens sent in and $15 for every million it writes, and you serve 50,000 requests a day.
1. Input cost per request: 2,000 × $3 ÷ 1,000,000 = $0.006
2. Output cost per request: 300 × $15 ÷ 1,000,000 = $0.0045 (Output tokens usually cost several times more than input tokens.)
3. Cost per day: $0.0105 × 50,000 = $525
Answer: About $525 a day, roughly $16,000 a month.

Then ask where to save: a shorter prompt, a cache for repeated questions, or sending easy requests to a cheaper model. These are example prices, so check your provider's current list. [Production AI](https://insideout.sh/courses/production-ai) covers routing, caching and budgets.

## 6. Show it with projects

1. **An assistant with a test set.** It answers questions from a set of documents, and you report its score on 50 real questions after every change.
2. **A retrieval system you measured.** Compare keyword, vector and combined search on the same questions, and show the numbers.
3. **An agent with limits.** A tool-using agent with a step budget, logged traces, and a clear way of stopping when it gets stuck.

Write up each one: what you tried, what the numbers said, what you'd do next. Interviewers read that write-up. Every insideout module has a [project](https://insideout.sh/projects) you build on your own machine.

## 7. The interview

AI engineer interviews usually mix a build round or take-home, a conversation about evaluation, design of a product with a model inside, questions on how models work, a coding round, and a talk about your past work. [The AI engineer interview page](https://insideout.sh/interview/ai-engineer) maps each round, with practice.

- **Build round.** Ship something small that works, and say how you would know it works.
- **Evals.** You'll be asked how you would measure a feature. Have a habit, with numbers.
- **System design.** A product with a model inside: retrieval, caching, fallbacks, cost and latency.
- **How models work.** Tokens, attention, context windows and sampling. [How large language models work](https://insideout.sh/blog/how-llms-work) is a quick refresher.

## Questions

**Do I need a degree to become an AI engineer?** No. Employers look for evidence that you can build and measure working systems. A computer science background helps, but projects with honest results and a clear write-up count for a lot.

**Do I need to learn how to train models?** You should understand how training works, so you know what fine-tuning can and can't do. Most AI engineering uses models someone else trained, adapted with prompts, retrieval and sometimes light fine-tuning.

**How long does it take to become an AI engineer?** For a working developer, three to four months at about ten hours a week covers the skills, then a month or two for projects. Getting hired also depends on the projects you can show.

**Should I learn Python or JavaScript?** Python first. It is the default for data, evaluation and most model tools. JavaScript or TypeScript is common for the product around the model, and easy to add later.

## Learn it properly

- [Training, validation and test sets](https://insideout.sh/courses/classical-ml/03) (Classical Machine Learning, lesson 3, free)
- [Attention as a soft lookup](https://insideout.sh/courses/transformers/02) (Transformers, lesson 2, free)
- [How a chat request is built](https://insideout.sh/courses/prompting/01) (Prompting, lesson 1, free)
- [Specifying a good output](https://insideout.sh/courses/evals/01) (Evaluation, lesson 1, free)
- [Keyword search with TF-IDF and BM25](https://insideout.sh/courses/search-retrieval/01) (Search and Retrieval, lesson 1, free)
- [Observe, decide, act](https://insideout.sh/courses/ai-agents/01) (Agents, lesson 1, free)
- [Routing requests between models](https://insideout.sh/courses/production-ai/01) (Production AI, lesson 1, free)
