Machine Learning Model Building Cheat Sheet
A general machine learning reference for problem definition, feature preparation, model creation, training, validation, metrics and interpretation.
What this reference covers
This general modelling reference moves from a business or research question to a measurable experiment. It covers target definition, prediction time, feature preparation, data splitting, baseline models, training, validation, metric comparison, error review and result interpretation.
Turn the question into a data contract
Specify the target, sample unit, prediction time, available fields and cost of errors before selecting an algorithm. Split data according to time, entity or stratification rules that match the way the model will be used. Fields created after prediction time are potential leakage.
Build a baseline and inspect failures
A simple model checks that the data path, split and metric are working. Add complexity only after the baseline is understood. Compare stability, resource cost and interpretability, and group error examples by business type rather than relying on one average score.
Maintenance note
Models can be affected by biased samples, concept drift, class imbalance and label errors. A high metric does not replace domain, privacy, monitoring and rollback review. Content review date: 2026-08-23.
Save to your cloud drive
Save the complete collection first so files remain together and are easier to access across devices.
Quark Cloud Drive
RecommendedSave Machine Learning Model Building Cheat Sheet to this cloud drive
Baidu Netdisk
Save Machine Learning Model Building Cheat Sheet to this cloud drive
Machine learning model-building study guide
Define the target and prediction time, fix the data boundary, establish an interpretable baseline and compare results with errors and costs in view.
Before you start
- Prepare a PDF reader and a labelled data set with basic source checks.
- Define the prediction target, time point, sample unit, available fields and error costs.
- Choose evaluation metrics that match the task and class balance.
Quick start
- 01
Write the target contract
State the target, prediction time, sample unit and success rule, and distinguish classification, regression, ranking or another task.
- 02
Split data and prepare features
Use a time, entity or stratified split, handle missing and categorical fields, and exclude information created after prediction time.
- 03
Run an interpretable baseline
Verify training, validation, metrics and error samples with a simple model before adding complex algorithms or features.
- 04
Compare and record
Compare metrics, stability, resource cost and interpretability on a fixed boundary, recording parameters, seed, data version and stopping reason.
Usage tips
- Feature engineering must respect the prediction time because future information can leak into training.
- For imbalanced classification, review recall, precision, specificity or PR curves in addition to accuracy.
- Group error samples by business meaning to reveal costs and data defects hidden by averages.
Troubleshooting and uninstall
Why is training performance strong but validation poor?
Check overfitting, duplicate entities, split rules, leakage features and feature count, then return to the fixed baseline.
Why do repeated experiments vary widely?
Fix the random seed and report repeated-run or resampling distributions, while checking sample size, class ratios and time drift.
Frequently asked questions
Where should machine learning model building start?
Start with target, prediction time, sample unit and evaluation criteria before preparing features or training a model.
Why establish a simple baseline?
A baseline verifies the data path, split and metric and shows whether a complex model provides a real improvement.
How can feature leakage be found?
Check when each field was created, whether it uses post-outcome information, whether preprocessing used all data and whether entities repeat across splits.
Why record error examples?
Errors reveal data defects, class boundaries, bias and business cost that a single average metric cannot explain.