The Ecosystem of Machine Learning Is Much Larger Than Algorithms

My algorithm is better than your algorithm! What gives? Well, it depends. The ecosystem that the algorithm must live in to deliver value is large, and it must be viewed as a whole in order to make sense of what is better and worse. Our industry is engaged in a debate over and the adoption of methods related to digitization, most of which come from machine learning or artificial intelligence, with many of these terms being used synonymously.


Before these methods can be used, data must be collected and aggregated in one place. This is the job of control systems and historians. Recently, some more software tools have entered the fray under the name “platform” and do the same thing in the cloud rather than on the premises. On one hand, we must have this collection and storage infrastructure in place. On the other hand, we must have confidence that this data is (1) more or less accurate in terms of measurement uncertainty, (2) correct regarding the physical phenomenon that it is supposed to characterize, and (3) representative of the system in that there is not some important quantity that is not being measured. We have expended much effort over decades to ensure most of this. It is not always the case, however. So it is important to review these points. These issues can be compared with a country’s road infrastructure on which we want to drive a car.

The algorithm that should provide some value must have access to this data collection, so we need to watch out for the necessary interfaces; this is not usually much of a problem. In the worst case, this means some work and some delay but not a true problem. The algorithm can be viewed like a car’s engine. It’s rather important, but it’s not a car yet.

It starts to get interesting once we have the infrastructure and the engine in place. Two important factors remain. The result of the computation has to arrive at the place where it can effect a real difference. This requires some more interfaces, protocols, and procedures to be in place. We can compare this to adding wheels and steering to the car’s engine. Finally, the car needs comfortable leather seats; this is a user interface that is easy and intuitive to use for a real user.

In real projects in our industry, the most difficult part is the wheels of the car. Adding the right procedures that allow the results of a computation to be used to change the way things have been done up to now requires a human organization to change the way it does business, usually in a radical way. Take, for instance, the use-case of predictive maintenance. So you have an algorithm that tells you that your compressor will get a blade tear in 5 days. Well done! Now what? Who is going to decide to power it down even though it’s still working? What spare parts do you need? How do you plan a repair before the equipment is broken? This procedure must be thought about carefully and must be in place. Once the message arrives and you have 5 days to do something about it, it is too late to make fundamental decisions. Additionally, do you trust the algorithm? Machine learning achieves very high accuracies, but nothing and no one is right all the time.

It is well and good to do pilot projects to determine whether some algorithm is capable of delivering this message with high accuracy and a sufficient amount of time in advance of the failure. This must be done, and it is great that our industry is actively engaged in it. Thank you, you know who you are. This, however, is not enough.

Whether you choose Algorithm A or Algorithm B or whether you pick Supplier X or Supplier Y is really a secondary choice. These technologies exist, and they work. The important decision is the strategic decision to change your organization to enable a technology of this type. Most of the work lies in the change management and setting up of the ecosystem.

If you ever change your mind about the algorithm, you can swap out the algorithm fairly easily because the ecosystem is ready. This is like bringing your car to the mechanic with a damaged engine. It’s annoying, takes a few days, and costs some money, but it is far cheaper than a new car.

There are plenty of cases in our industry and related industries of chemistry and metallurgy where machine learning technologies have been proven to work great but have been ignored by the people on the ground and, thus, delivered no actual value. Let this be a cautionary tale to us all—to the owner/operators in order to think about the ecosystem in advance and detail; to the suppliers in order to offer advice and assistance in building the ecosystem together with the end users.

Too much emphasis is being put on the algorithms, with about 95% of the literature discussing technical issues of mathematical modeling. This discussion hides the problems of getting data and putting the methods in an organization made of people who must make some decision or perform some action on the basis of the analysis. Machine learning delivers information and actionable insights. Value is generated by people using this information.


Don't miss out on the latest technology delivered to your email monthly.  Sign up for the Data Science and Digital Engineering newsletter.  If you are not logged in, you will receive a confirmation email that you will need to click on to confirm you want to receive the newsletter.