Machine Learning Fundamentals

How to Use This Book

Read linearly on the first pass. The book is designed to build intuition before algorithm accumulation.

Environment Setup

Use a Python environment with NumPy, plotting tools, testing support, and scikit-learn available for comparison once from-scratch implementations are complete.

Reading Strategy

Implement the algorithms, then compare them against library versions, then evaluate them as part of full projects.

The Machine Learning Workflow

A data analyst at an insurance company builds a model to predict which claims are fraudulent. She downloads a dataset, imports a gradient boosting library, calls .fit() and .predict(), and reports 97% accuracy to her manager. The manager asks three questions: "What does 97% mean when only 1.5% of claims are actually fraudulent? Did you check whether the model just predicts 'not fraud' every time? And how do you know it will work on next month's claims?" The analyst does not have answers. The library call succeeded. The project failed.

This is the most common failure mode in applied machine learning. The model ran. The number looked good. But the analyst skipped the workflow that makes the number meaningful: problem framing, data splitting, metric selection, validation design, and iterative evaluation. Without that workflow, even a correct implementation produces results that no one can trust.

This chapter establishes the workflow you will use for every model in this book. Algorithms change from chapter to chapter. The workflow does not.

• Define what machine learning is and identify problems where it is the right tool
• Map the full supervised learning workflow from problem framing through deployment
• Explain why metric choice and validation design determine whether results are trustworthy
• Distinguish training, validation, and test sets and explain the role of each
• Identify data leakage, weak evaluation, and other structural failures before they corrupt results
• Describe the algorithms and techniques covered in this book and how each fits the workflow

What Machine Learning Is and Is Not

Machine learning is a method for building systems that improve their behavior on a task by learning from data, rather than following explicitly programmed rules. That definition is broad on purpose. It covers predicting house prices from square footage, classifying emails as spam or not spam, segmenting customers into groups based on purchasing behavior, and hundreds of other tasks. The unifying idea is that the system extracts patterns from data and uses those patterns to make decisions about new, unseen inputs.

The Core Idea: Generalization

The goal of machine learning is not to memorize the data you have. It is to learn patterns that generalize to data you have not yet seen. A model that perfectly memorizes its training data but fails on new inputs is useless in practice. This distinction between memorization and generalization is the central tension in every machine learning project, and it will reappear in every chapter of this book.

Consider a concrete example. You have records of 10,000 houses that sold last year, including features like square footage, number of bedrooms, neighborhood, and sale price. You want to predict the sale price of houses that will sell next month. A model that memorizes the exact prices of the 10,000 houses is unhelpful — those houses already sold. What you need is a model that learned the relationship between features and price well enough to make accurate predictions on houses it has never seen.

Generalization is the ability of a model to perform well on new, unseen data — not just the data it was trained on. Every design decision in the ML workflow exists to measure and improve generalization.

Supervised, Unsupervised, and Reinforcement Learning

Machine learning divides into three broad paradigms based on the type of feedback the learning algorithm receives.

Supervised learning uses labeled data — each input comes paired with the correct output. The algorithm learns a mapping from inputs to outputs by minimizing the difference between its predictions and the true labels. Regression (predicting a continuous value) and classification (predicting a category) are the two main supervised tasks. Chapters 2 through 7 of this book focus on supervised learning.

Unsupervised learning works with unlabeled data. The algorithm finds structure — clusters, patterns, compressed representations — without being told what the "right answer" is. Chapter 8 covers clustering and Chapter 9 covers dimensionality reduction.

Reinforcement learning involves an agent that learns by interacting with an environment, receiving rewards or penalties for its actions. Reinforcement learning is outside the scope of this book, but it is worth knowing the category exists.

Most applied machine learning is supervised learning. You have historical data with known outcomes, and you want to predict outcomes for new cases. This book reflects that reality.

When Machine Learning Is the Wrong Tool

Machine learning is not the right tool for every problem. Recognizing when not to use it is as important as knowing how to use it well.

When the rules are known and fixed. If a business rule says "flag any transaction over $10,000 for review," you do not need a model. You need an if statement. Machine learning is appropriate when the decision boundary is complex, unknown, or changes over time.

When you have no data. Machine learning requires data — usually a substantial amount. If you are making a decision based on three observations and expert judgment, that is not a machine learning problem. It is a reasoning problem.

When the cost of errors is catastrophic and unverifiable. If the consequences of a wrong prediction are severe and you have no way to validate the model's behavior before deployment, a rule-based system with human oversight may be more appropriate than a learned model.

When a simpler method works. If a linear trend line explains 98% of the variance in your data, you do not need a neural network. The simplest model that solves the problem is usually the best model, because simpler models are easier to debug, interpret, and maintain.

Reaching for machine learning before defining the problem. The question "Can we use ML for this?" is backwards. The right sequence is: define the decision, determine what prediction would support that decision, assess whether data exists to learn that prediction, and then choose the simplest method that works. Machine learning is a tool in the workflow, not the starting point.

Models as Tools for Generalization

A machine learning model is a mathematical function that maps inputs to outputs. The function has parameters — numbers that the learning algorithm adjusts during training to minimize prediction error. Different model families (linear models, decision trees, neural networks) define different mathematical structures, but they all share the same purpose: learn a mapping that generalizes beyond the training data.

The word "model" can mislead. In everyday language, a model is a simplified representation of reality. In machine learning, a model is a specific, trained function with fixed parameters. When someone says "the model predicts $350,000 for this house," they mean a concrete function was evaluated with concrete inputs and produced a concrete output. There is nothing vague or approximate about what the model does. The uncertainty lies in whether that output is correct — whether the model's learned patterns generalize to this particular house.

The Full Workflow from Data to Evaluation

Stage 1: Frame the Problem

Every ML project starts with a question, not a dataset. Framing means defining four things before writing any code:

1. What decision will the model support? A model that does not inform a decision has no value. "Predict customer churn" is not a decision. "Identify customers likely to cancel in the next 30 days so the retention team can contact them" is a decision.