A statistical model is the use of statistics to build a representation of the data and then conduct analysis to infer any relationships between variables or discover insights. Machine learning, on the other hand, is the use of mathematical or statistical models to obtain a general understanding of the data to make predictions. Still, many in the industry use these terms interchangeably. While some may not see any harm in this, a true data scientist must understand the distinction between the two.
Statistical Modeling
Statistical modeling is a method of mathematically approximating the world. Statistical models contain variables that can be used to explain relationships between other variables. Hypothesis testing and confidence intervals, for example, are used to make inferences and validate hypotheses.
The classic example is regression, in which one or several variables are used to find the effect of each explanatory variable on the independent variable.
A statistical model will include sampling, probability spaces, assumptions, and diagnostics, for example, to make inferences.
Statistical models are used to find insights given a particular set of data. Modeling can be conducted with a relatively small set of data just to try to understand the underlying nature of the data.
Inherently, all statistical models are wrong, or, at least, not perfect. They are used to approximate reality. Sometimes, the underlying assumptions of the model are far too strict and not representative of reality.
Machine Learning
Machine learning is the method of teaching a computer to learn like humans. Computers, of course, can have far greater capacity than the human mind, so, when an enormous amount of data has been collected that is beyond a normal person’s comprehension or capacity to understand the patterns in the data, the computational and storage power of computers can outshine a human.
Simply, machine learning is used to make predictions, and its performance can be assessed by how well it generalizes to new data that is has not learned yet.
Cross-validation is conducted to validate the integrity of the data to make sure the model does not overfit (memorize) or underfit (not enough data to learn) the data.
The data is cleaned and organized in a manner that the machine can understand. No, or very minimal, statistics go into this process.