Feature Engineering: Preparing High-Performing Variables for Machine Learning
Feature Engineering is an advanced-level course of 6 lessons (about 3 h 35 min), with 13 exercises and 18 quiz questions. Prerequisites: Supervised Learning: Regression and Classification, Data Cleaning and Preparation. Exercises run in the browser, with nothing to install.
- Data science
- Advanced
- 6 lessons
- 3 h 35 min
- 13 exercises
- Exercises run in the browser
The course in brief
This advanced course teaches you to turn raw, heterogeneous data into features that machine learning models can use. You handle numerical, categorical, and temporal variables, deal with missing values, build complete pipelines with ColumnTransformer, create business features without data leakage, and select the variables that are really useful with scikit-learn and pandas.
What you will be able to do
- Transform numerical variables (log, power, discretization, polynomials) according to the target model
- Encode categorical variables with OneHotEncoder, OrdinalEncoder, and TargetEncoder
- Impute missing values and exploit the information carried by absence
- Build a complete pipeline for heterogeneous data with ColumnTransformer
- Create temporal, cyclical, and business features without data leakage
- Select features and interpret permutation importance
Syllabus: 6 lessons
- Correct the skewness of a variable with log1p or PowerTransformer
- Discretize a variable with KBinsDiscretizer
- Create interactions and polynomial terms with PolynomialFeatures
Numerical transformationsDiscretizationInteractions and polynomials
- 02
Encode Categorical Variables
- Choose between one-hot, ordinal, and target encoding
- Handle unknown categories and rare categories
- Use TargetEncoder without data leakage
- 03
Handle Missing Values
- Impute missing values with SimpleImputer and KNNImputer
- Preserve the "missing value" information with add_indicator
- Choose between imputation and models that handle NaN natively
- 04
ColumnTransformer: Pipelines for Heterogeneous Data
- Apply different preprocessing depending on column type with ColumnTransformer
- Assemble a complete pipeline from raw data to prediction
- Inspect the features produced with get_feature_names_out and set_output
- 05
Create Business and Time-Based Features
- Extract useful information from a date
- Encode cyclical variables (hour, month) with sine and cosine
- Integrate custom business features into a pipeline with FunctionTransformer
- 06
Select the Useful Features
- Apply filter methods (VarianceThreshold, SelectKBest)
- Select via a model (SelectFromModel) and by permutation importance
- Place selection in the pipeline to avoid any leakage
Filter-based selectionModel-based selectionPermutation importance
Final project
Pipeline to Predict Time-to-Sale of Real Estate Listings
Use numpy and pandas to generate a dataset of 5,000 realistic real estate listings: publication date, surface area, number of rooms, price, neighborhood (about a hundred levels), property type, floor, presence of a balcony, an ordered energy rating, with plausible missing values and a target "sold in under 60 days". Build a complete pipeline: business features (price per m², deviation from the neighborhood's average price computed without leakage), time-based and cyclical features, encodings suited to each variable, imputation with indicators, feature selection. Compare a linear model and a HistGradientBoosting by cross-validation and analyze the permutation importances.
Deliverables
- The data generation script and the complete scikit-learn pipeline that accepts raw data
- An ablation table showing the gain from each family of features (cross-validation)
- A chart of permutation importances on a validation set
- A note explaining each encoding choice and the precautions taken against leakage
Assessment criteria
- All learned transformations are in the pipeline (no leakage)
- Encodings consistent with the nature of the variables and the model
- Relevant business and time-based features that are computable at prediction time
- Rigorous comparison of variants by cross-validation
- Pipeline robust to missing values and unknown categories
Frequently asked questions
What is feature engineering?
It is the set of transformations that make raw data usable and informative for a model: numerical transformations, category encoding, handling of missing values, creation of business variables, and selection of the most useful ones.
Is feature engineering still useful with gradient boosting or deep learning?
Yes. Powerful models reduce the need for some transformations, but business features (ratios, aggregates, temporal information) and rigorous handling of categories and missing values often remain decisive on tabular data.
How do you avoid data leakage during feature engineering?
By putting every transformation that learns parameters inside a scikit-learn pipeline, by using TargetEncoder rather than a hand-computed target mean, and by building aggregates only from information available at prediction time.
What are the prerequisites for this course?
You need to know scikit-learn's supervised models (the Supervised Learning course) and be comfortable with pandas for handling DataFrames, dates, and groupings.
This course, told for your job
Say what you want to be able to do: “Feature Engineering” takes its place in your path, with examples from your field. The concepts stay those of the course.