Machine Learning
What is gradient boosting?
Gradient boosting is a machine learning method that builds many small decision trees in sequence, each one correcting the errors of the ones before it.
How it works
Train a shallow decision tree on your data. It gets a lot wrong. Train a second tree specifically on the residual errors of the first. Then a third on what the first two still miss. Add them together with small weights. After a few hundred rounds the ensemble captures interactions that no single tree - and no linear model - would find.
The three implementations you will meet
XGBoost is the widely-deployed default with the most tooling around it. LightGBM trains faster on large tabular datasets and handles high-cardinality categorical features well. CatBoost handles categorical variables natively with less preprocessing. For most business tabular problems the differences in final accuracy are small - pick one and spend the time on features instead.
Why it beats an LLM on money decisions
Gradient boosting on tabular data is deterministic, auditable, cheap to run at scale, and gives you per-feature attribution for every prediction. An LLM asked to score a customer will produce a plausible number with no traceable derivation and no stability guarantee between runs. When the score decides where budget goes, that difference matters more than anything else.
What it needs from you
Clean tabular history with a real time dimension, and a label you can actually define. Six months is a workable floor for most retention problems. It does not need deep learning infrastructure, GPUs, or a data lake. A well-featured gradient boosting model on a laptop beats a badly-featured neural network on a cluster in almost every business context.
Common failure mode
Target leakage - a feature that encodes the answer. If your training data includes 'account closed' as a feature for predicting churn, the model will look brilliant offline and useless in production. Every feature has to be computable at the moment you would actually make the prediction, not after the outcome is known.
We build automation systems for small teams. Free audit call to map your specific workflows - no pitch, just a plan.