When to use Excel vs. when to use ML.

I recently caught up with a CTO friend who asked, “Can Machine Learning help me?” He then described a scenario that occurred once per year, had scant historical data, and had no way of quantitatively measuring whether success was achieved. “No,” I replied, “A spreadsheet and a BI intern is better off handling this for you.” Even though Machine Learning (ML) is a powerful tool, not all problems need ML solutions.  So what are the characteristics of a situation where Machine Learning can really excel? (bad pun intended). 

Data Rich Environment.

Yes, machines can play GO better than humans now, but there are certain things that they aren’t good at (yet). One of those things is intuition in low-data environments. Knowing what to do when a situation hasn’t been encountered, there is no previous data to guide what makes a good decision. Humans are really good at this. Some call it creativity or gut instinct.

Now on the flip side, Machines are good at parsing through mounds of data to find patterns and extrapolating what the following possible outcomes are likely to be. Think about the dashboards your team or company uses. Notice how they keep getting increasingly complicated with tons of data points? All that data is supposed to help teams make better business decisions or insights, but the truth is our brains can only handle so much of that (4 data points at once). So here is where Machine Learning has a distinct advantage. ML can (without sleep, food, or holidays) crunch through data and find patterns that can help a business achieve its goals. If you have a problem that throws off a lot of data and have to make sense of that data to figure out what to do next, this is a great Machine Learning use case. 

‍

Operationally Relevant. 

In business terms, this is often referred to as “Is the juice worth the squeeze?” Until recently, the cost of Machine Learning implementation didn’t make sense for many business problems. Because solving these problems would require a business to build Data Science teams (which is getting increasingly difficult), buy expensive software for those teams, and then leverage a lot of other resources to build out data pipelines to feed the ML infrastructure. Only the largest problems - often at large companies - passed the “Operationally Relevant” test.

One CIO recently told me that he wouldn’t use Machine Learning to touch any problem that didn’t generate at least $5 Million of savings or lift. It just wasn’t worth it. And even then, he preferred $20 Million and above problems.  But luckily, times have changed, and tools (oh hi 👋) have made it possible to make the ROI of Machine Learning work for much smaller but still relevant use cases. When doing the math, businesses must remember to include their own teams' time as an expense. Here is where Machine Learning can get an edge up because ML brings automation. Often teams forget to calculate their own time and opportunity costs when using spreadsheets.

‍

Recurring Problems or Decisions. 

Machine Learning succeeds on problems or decision points where it can find patterns. For patterns to occur, these decision points need to happen regularly. If you have a situation that rarely occurs, there likely isn’t a lot of data available, so pattern recognition becomes challenging. Also, if the decision point’s outcomes are far removed from when the decision occurred, it’s probably better to use Excel. For example, if it takes a year to learn whether your solution worked or not, this is likely a good place for a spreadsheet (and a good memory). However, if your decision point occurs frequently, and your team is spending a lot of time crunching through data every day, week, or month trying to make the same decision over and over again, Machine Learning can add a lot of efficiencies.

Machine Learning has a lot of benefits to organizations - automation, cost reduction, profitability - but first, the best use cases have to be identified where Machine Learning can be successful. Once you apply the simple three-pronged approach above, you can confidently answer the question, “Excel or ML?”

‍