insideout Get Pro

08 · Decision trees

Pro

Classical Machine Learning, lesson 8

Decision trees

Split the rows one question at a time until each group mostly agrees.

"Training, validation and test sets" (overfitting) and "Measuring a classifier" (accuracy against a do-nothing rule). The churn customers are the ones from "Logistic regression".

Part 1. A model made of questions

Logistic regression adds up evidence from every feature for every customer. A decision tree works more like a support script: ask one question, then choose the next question by the answer, until you reach a verdict.

Here's a real one, grown by scikit-learn on 2,000 churn customers made by this lesson's code. It's been told to ask at most two questions.

Example 1.1

A churn tree, two questions deep

  1. months ≤ 17.5?

    The first question splits newer customers from older ones. The tree chose it because it separates churners from stayers better than any other single question.

  2. yes, newerbill ≤ $60.78?
    no, oldermonths ≤ 30.5?

    Each side asks its own second question. Newer customers are split by their bill, older ones by tenure again.

  3. newer, cheaper121 of 347 churned→stays (35%)
    newer, pricier250 of 384 churned→churns (65%)

    At the bottom, each group predicts the majority of its training customers, and its churn share serves as a probability.

  4. 18 to 30 months110 of 546 churned→stays (20%)
    over 30 months30 of 723 churned→stays (4%)

    Long-standing customers rarely leave. The bill is never asked about on this side: the tree found it mattered only for newer customers.

Decision tree

A model that sends each row through a series of yes-or-no questions about its features, where each question depends on the answers before it. It predicts from the training rows that ended up in the same place.

Leaf

A group at the bottom of a tree, where the questions stop. A classification leaf predicts its majority class, with its class shares as probabilities. A regression leaf predicts the average of its rows.

Your turn

Using the tree above, what does it predict for a customer of 24 months with a $110 monthly bill?

24 months answers no to "months ≤ 17.5?" and yes to "months ≤ 30.5?", which lands in the 18-to-30-month leaf: 110 of 546 churned, about 20%. The bill question only exists on the newer customers' side, so a $110 bill changes nothing here. A tree never needs to have seen the exact customer; it only needs their answers to its questions.

Part 2. Choosing a question

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Choosing a question
  2. Growing until it memorises
  3. What trees are good and bad at
See Pro plansI have Pro: sign in
Next lessonRandom forests and gradient boostingMany trees voting, or small trees that each fix the last one's mistakes. Start lesson 9

Classical Machine Learning overviewSpotted a mistake? Tell us.