Feature Engineering for Traders

A production-oriented feature engineering system that spans price, order flow, cross-asset, sentiment, and model-specific feature pipelines.

How to Use This Book

Most traders who use machine learning spend their time swapping model families — trying random forests, then gradient boosting, then neural networks — while feeding all of them the same mediocre features. The returns from better features almost always exceed the returns from better models. This book teaches you to design, evaluate, and prune feature stacks that actually improve trading models without drifting into leakage, redundancy, or feature bloat.

The scope starts with foundational price and volatility features and moves through order flow, microstructure, institutional-activity detection, cross-asset signals, and model-specific pipeline design. By the end, you will have a feature engineering workflow that produces traceable, testable inputs for any model family you choose to deploy.

Read linearly on the first pass. Later chapters assume you already understand the difference between useful features, descriptive clutter, and silent leakage.

Who This Book Is For

• Traders or ML practitioners who already know basic model training
• Readers who want to improve model quality through stronger feature design rather than model shopping
• Practitioners comfortable with tabular, time-series, and optionally tick-level data

Prerequisites

• Python data workflows (pandas, numpy, basic scripting)
• Familiarity with supervised learning basics
• Basic time series and backtesting literacy
• Comfort with tabular and time-series data

Environment Setup

Use a Python environment with standard data, statistics, visualization, and modeling libraries.

• Data — pandas, numpy, polars (optional)
• Modeling — scikit-learn, lightgbm, xgboost
• Visualization — matplotlib, seaborn
• Statistics — statsmodels, scipy
• Feature Tools — ta-lib or equivalent, feature-engine

Keep raw data, intermediate feature tables, feature-generation code, and evaluation reports organized so every feature can be traced back to the transformations that created it.

Chapter Roadmap

• 1. Foundational Financial Features — Price, volatility, cross-asset features, normalization — Core feature layer for trading models
• 2. The Advanced Feature Stack — Order flow, spectral, liquidity, text-derived signals — Expanded feature set with complexity control
• 3. Selection, Importance, and Stacking — Pruning, interpretation, feature organization — Disciplined feature selection workflow
• 4. Order Flow Analysis Deep-Dive — Real-time order flow and microstructure signals — Flow-based ML signal pipeline
• 5. Advanced Order Flow Features — VPIN, toxicity, hidden liquidity, higher-order flow — Institutional-quality flow features
• 6. Institutional Activity Detection — Flow-pattern-based institutional behavior detection — Honest institutional activity framework
• 7. Cross-Asset and Temporal Feature Engineering — Multi-asset, time structure, regime-aware features — Generalized cross-asset feature system
• 8. Feature Engineering for Different Model Families — Pipelines tailored to trees, linear, deep, ensembles — Model-specific feature pipeline templates

Reading Strategy

Work through examples and exercises rather than treating the book as a catalog of formulas. Chapters 1 through 3 establish the core feature-engineering framework. Chapters 4 through 6 focus on order flow and institutional-style flow signals. Chapters 7 and 8 generalize the feature layer across related assets, time structure, and model families. Every feature introduced is evaluated against the question: does this actually help the model, or does it just look informative?

Foundational Financial Features

A quant researcher at a mid-sized fund spends three weeks building a gradient-boosted model to predict next-day equity returns. The model uses closing prices, volume, and a handful of technical indicators pulled from a charting library. On the validation set, the results look reasonable. In live trading, the model drifts within weeks. The researcher discovers the problem is not the model. The problem is that closing prices, raw volume, and default indicator settings carry far less usable information than the researcher assumed. The features looked like data. They were not yet information.

This chapter builds the first layer of a defensible feature set: the foundational features that transform raw market bars into representations a model can use across instruments, time periods, and volatility regimes. Every feature introduced here must earn its place by improving what the model can distinguish, not by adding columns to a dataframe.

• Explain why raw OHLCV data is insufficient for robust trading models
• Construct derived price features including mid-price, typical price, and VWAP
• Apply volatility-aware normalization to make features comparable across regimes
• Transform technical indicators into structured model inputs rather than using them as trading signals
• Add cross-asset context features while controlling for time-alignment leakage
• Evaluate whether a foundational feature belongs in your baseline set

Beyond the Close: Why Basic Bars Are Incomplete

Most trading models begin with OHLCV bars: open, high, low, close, and volume. These five fields summarize what happened during a time interval, but the summary is lossy. A model trained on raw closing prices learns price levels, not price behavior. A model trained on raw volume learns absolute activity, not relative activity. The distinction matters because markets change regimes, instruments trade at different scales, and the same closing price carries different meaning in calm and volatile periods.

Consider two trading days for the same equity. On both days, the close is $150. On the first day, the high was $151 and the low was $149. On the second day, the high was $158 and the low was $143. The closing price is identical, but the intraday range, the implied volatility, and the position of the close within the range are radically different. A model that sees only the close treats these days as equivalent. A model that sees derived features can distinguish them.

The goal of foundational feature engineering is to extract this kind of structural information from raw bars and present it in a form that a supervised learning model can use.

Foundational features are the first layer of engineered inputs derived from raw market data. They capture price structure, volume context, and volatility state. Every later feature layer in this book builds on top of this foundation.

What Raw Bars Contain and What They Miss

A standard OHLCV bar provides five numbers per time interval. From these five numbers, a model can learn:

• the price level (from the close or open)
• the absolute range of prices (from high minus low)
• the absolute volume traded

What a model cannot learn directly from raw bars includes:

• where the close sits relative to the day's range
• whether volume is elevated or depressed relative to recent history
• whether the range is wide or narrow compared to the asset's typical behavior
• whether the bar's price action is consistent or contradictory (for example, a close near the low after opening near the high)

Each of these missing pieces is a feature waiting to be built. The next sections construct them systematically.

Common Failures from Raw-Bar Dependence

The consequences of using raw bars without derived features show up in specific, predictable ways:

• Scale confusion. A model trained on raw closing prices for one equity will assign different weight to the same percentage move for a different equity trading at a different price level. A $1 move on a $50 stock is 2%. A $1 move on a $500 stock is 0.2%. Without normalization, the model treats both identically.

• Regime blindness. A model trained on raw volume during a calm period will interpret normal volume during a volatile period as unusually high. The model is comparing absolute levels across regimes where the baseline has shifted.

• Missing intrabar information. Two bars with the same open and close but different highs and lows contain different information about participant behavior. A wide-range bar that closes at the open signals indecision or a fight between buyers and sellers. A narrow-range bar that closes at the open signals agreement on price. Raw OHLCV captures the range but does not present it in a form that a model can use directly for this inference.