Engineers make data-driven decisions on a daily basis. It is vitally important that the average engineer is sufficiently competent at gathering good data and properly interpreting its meaning. But most engineers are not equipped to handle data at scale. Most engineers do have neither the background nor the knowledge necessary to scale data pipelines or automate their efforts. This often results in mass data gathering but data rotting away with little to no use. The data will only be drug out when something goes wrong with a piece of equipment or batch in a process. Either engineers need to step up their game or data scientists and data engineers need to find their way into these teams.
Plant level engineering is one area data scientists are rarely found. In marketing and business functions, data scientists are found frequently in big companies. But, for whatever reason, plant engineering rarely includes engineers, statisticians, or even computer scientists capable of building data pipelines and analyzing data at scale. This is shocking considering the sheer amount of data gathered by sensors at the plant level. It seems that, in many places, the only use for data from these sensors is triggering alerts when the process values exceed some threshold.
Increasingly, companies are paying others to come in and engineer solutions for automating their data gathering and analytics. But is spoon feeding engineers answers out of thin air really solving any problems? Are these answers really that relevant in a constantly changing environment? And, if not, is Company A really prepared to bring in Company B every time a new model needs to be built or an old model needs to be refined?