Gradient Boosting Is Still the Answer for Tabular Data

Gradient Boosting Is Still the Answer for Tabular Data

Deep learning has not displaced gradient boosting on tabular data, and the candidate who reaches for a transformer on a fraud detection problem is telling the interviewer something about their judgment.

Why boosting keeps winning

Tabular data has properties that suit trees and frustrate neural networks: heterogeneous feature types, irregular decision boundaries, missing values with meaning, and no spatial or sequential structure to exploit. Trees handle all of that natively. Neural networks need it engineered away first.

Boosted models also train in minutes, tune predictably, and give you feature importance for free, which matters more than a decimal place of AUC when someone has to explain a declined transaction.

When to reach for a network anyway

  • High-cardinality categoricals where learned embeddings beat target encoding
  • Genuinely multimodal problems mixing tabular data with text or images
  • Very large datasets where the boosting curve has flattened
  • Transfer from a related task that already has a pretrained model

What this means in an interview

Propose boosting as the baseline, say why it fits the data, then name the specific condition that would make you change your mind. That answer reads as experience. "I would use a neural network" reads as a default.

The follow-up is usually about the imbalance, the threshold, or the cost matrix, and that is the real question anyway.

Keep reading

What a Tailored Resume Should Actually Mean
What a Tailored Resume Should Actually Mean

Not keyword stuffing, and not a new resume per job. Reordering the truth, with the numbers attached.

Scaling Laws, in the Arithmetic You Will Actually Use
Scaling Laws, in the Arithmetic You Will Actually Use

Bigger model or more tokens? The question has a numerical answer, and it comes up in interviews more than people expect.

The ML Interview Loop, Round by Round
The ML Interview Loop, Round by Round

Five rounds, five different things being graded. Most candidates prepare for two of them and are surprised by the rest.