Start at the first word.
step 1 of 8
1Watch it work
Before any code, you see the idea run on real Wendle text, one step at a time, with every step in plain words.
First up: bring Echo, an old talking machine, back to life.
Know GPT inside out.
For engineers who want to understand AI, not just use it. You build every part yourself, from a word counter to a small transformer you trained, in real Python, right in your browser.Build every part yourself, in real Python.
Free to start. No setup, no GPU. Python runs right in the page.
The papers you'll rebuild.
Eight of the landmarks on the way from counting to ChatGPT, out of the many papers you rebuild here. Each unit hands you the problem the authors had, and you build their answer. Tap or hover a paper to see where.
- 1948 A Mathematical Theory of Communication C. E. Shannon Unit 1 Counting what comes next
- 2003 A Neural Probabilistic Language Model Bengio, Ducharme, Vincent, Jauvin Unit 4 The first neural language model
- 2013arXiv Efficient Estimation of Word Representations in Vector Space Mikolov, Chen, Corrado, Dean Unit 5 Words as places on a map
- 2014arXiv Neural Machine Translation by Jointly Learning to Align and Translate Bahdanau, Cho, Bengio Unit 9 Looking back: attention
- 2016arXiv Neural Machine Translation of Rare Words with Subword Units Sennrich, Haddow, Birch Unit 6 Cutting words into pieces
- 2017arXiv Attention Is All You Need Vaswani, Shazeer, Parmar, et al. Unit 13 The transformer
- 2021arXiv LoRA: Low-Rank Adaptation of Large Language Models Hu, Shen, Wallis, et al. Unit 20 Small changes, big effect
- 2023arXiv Direct Preference Optimization Rafailov, Sharma, Mitchell, et al. Unit 19 Learning manners
Finish a sentence with Echo.
This is the first Echo, the one you build in Unit 1. It has read the 61 notices on Wendle's harbour board and counted which word comes after which. That's all it knows. Pick a word, then let it roll the dice.
ECHO
model 1 · counting
Or start with
After the, Echo has seen
Click a word to write it yourself. Press Roll to let Echo choose.
See it, build it, keep it.
Every lesson is short and ends with something that runs. Here is the loop, using one real lesson from Unit 1: counting pairs.
def count_pairs(words):
pairs = {}
for a, b in zip(words, words[1:]):
Returns a dictionary
pairs["the"]["ferry"] is 5
The last word isn't dropped
RunNot run yet
2Build it
Write the real thing in Python, in the browser. Echo's checks tell you exactly what's off, like "the last pair never gets counted".
1pairs = {}
2for a, b in zip(words, words[1:]):
3 pairs.setdefault(a, {})
4 pairs[a][b] = pairs[a].get(b, 0) + 1
a
'the'b
'fog'pairs[a]
{'fog': 1}3Trace yours
Step through your own code line by line and watch the variables change. It's how you find the bug yourself, instead of being told.
Review card · back in 3 days
What does this print?
pairs["the"].get("cat", 0)
Answer
2
"the cat" shows up twice in the notices.
4Keep it
New ideas get a name only after you've used them. Each one goes in your notebook and comes back as a review card, so it stays with you.
Rebuilt the way it was invented.
Echo gets rebuilt era by era: counting, then neural nets, then attention, then the transformer. Each rebuild fixes a real problem with the last one, so every new idea shows up with a reason.
The WorkshopFree
Rusty on the maths? A few short units on dictionaries, NumPy, chance, vectors and slopes. A short placement quiz skips what you already know.
The Lighthouse
Start hereHow language models work
- From counting pairs to self-attention
- Your own tokenizer, trained on Wendle's text
- A mini-GPT you train in the browser
The School House
How models are made
- Clean a messy web crawl into a dataset
- Fine-tune, then teach manners with DPO
- LoRA, quantization and the cost of a run
The Harbour
Building with models
- Search by meaning, then RAG on a real rulebook
- Tool calls, and when not to use an agent
- Ship an app with a test report
Every unit, in order
Pick a machine to take apart.
Each one is a machine you rebuild yourself, part by part, in Python that runs in the page. Open one to see every unit, what you build in it, and the paper or system it comes from.
Five things you made, yours to keep.
Real work you can open, run and show someone in an interview.
echo-gpt
4 layers · 4 heads · your tokenizer
Data card
Pages in10,000
Junk removed2,840
Near-copies1,112
Clean pages6,048
When does the last ferry leave?
Beforethe ferry the ferry is the
AfterAt 9pm, unless the fog is thick.
Test report46 / 50
4 misses, all prompt injection. Fixed in v2.
Priya Raman
Start free. Pay when it clicks.
Unit 1 of Rebuilding Echo is free, and you don't need a card. When you want more, every plan opens everything. You only choose how you'd like to pay.
Things people ask first.
Something else? Write to hello@insideout.sh. A person answers.
Do I need a machine learning background?
No. You need to be comfortable writing a loop and a function. Every idea starts with a small example you can see, and only gets its proper name after you've used it. If the maths is rusty, the free Workshop covers the bits you need: chance, vectors and slopes.
How much Python do I need?
Lists, dictionaries, loops and functions. If you can count the words in a file, you're ready. NumPy gets introduced in the Workshop before you need it, and we build our own small version of autograd instead of starting with a big framework.
Is it based on a university course?
The order of topics follows a graduate course on large language models, so if you're taking one, the units line up and Course Mode will match your term. All the explanations, examples, stories and exercises are our own. No slides or figures are reused.
Can my company pay for it?
Yes. Most learning budgets cover it. The receipt from checkout is all you need to submit, and we can add your company's details to it.
What if it isn't for me?
Email us within 30 days of paying and we'll refund all of it. You don't need a reason. Cancelling is one click, and you keep your code and portfolio page either way.
Does it work on my phone?
Reading, watching the demos and doing review cards all work well on a phone. Writing code is much nicer on a laptop, so that's where we'd suggest doing the building.
Where does the Python actually run?
In your browser, using Pyodide. Nothing to install, and your code never leaves your machine while it runs. Later units train small models in the page too. They're small on purpose, so a normal laptop is enough.