SERVICENOW, INC. Nearly 9 in 10 enterprises increased annual spending on machine learning and AI projects in 2021, according to a DataRobot report. But their potential ROI is far lower than it could be, says Eric Siegel, a machine learning expert and author of "Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die."
Most projects stall out before deploying machine learning models, according to Siegel's January 2022 survey of data scientists.
In a recent interview with Workflow, Siegel explained why leadership, not technology, is to blame for the high failure rate, and what CIOs and other senior executives can do to reverse the trend.
Because they're typically viewed as technology projects when they should be seen as organizational ones. The point is to improve a company's operations, not to flex high-tech muscles.
Percentage of machine learning models that a majority of data scientists say are ever deployed.
There's a fetishization that happens with machine learning, and that applies to even the most hardcore technical experts. They're trained to download the software and start making models without stopping to think, "Wait, this is a business endeavor; what is this project really supposed to achieve?" They may consult with their business counterparts, but not nearly to the depth that is required.
These should be treated as projects meant to improve operational performance, where machine learning is but one key component. Ultimately, it's up to executives to identify the business goals and work with the data scientists on the best ways to achieve them.
More semi-technical understanding may be necessary, but it's actually a myth that machine learning is hard to understand. It's just a probability calculator: "Which customer is more likely to do X if you do Y?"
It's like driving a car. You don't need to know how an internal combustion engine works, but you need to know how to operate a steering wheel, the concepts of momentum and friction, and have a destination in mind.
You need to start by defining your business objectives. But you also need to be very specific about what your ML models are designed to predict.
It's not enough to ask which customers are likely to cancel. Instead, you should ask which customers who've been around for three years will decrease their spend by a specific amount over the next four months. The business objective can be general, but the prediction goal and deployment particulars have to be much more specific. And you need to agree on the metrics you'll use to measure success.
Once you've gotten those predictions, what are you going to do differently as a result? What operational change will you embrace that will make an impact? For example, if you've got a team of auditors that have been doing fraud detection the same way for 20 years, how are you going to convince them to do things a new way? They may not be aware of how data-driven probabilities work.
It's not horribly technical, but they may be set in their ways.
One big reason why ML projects don't get deployed is because the line of business managers are used to doing things a certain way and don't really want to change.
That depends on the organization. Some projects are led bottom-up, some are top-down, some a combination of the two. Change happens in all sorts of funny ways. The existence of a CIO, CTO, chief analytics officer, or chief data officer certainly speaks to the idea of someone leading this process, but I can't say there are that many people in those roles who are really getting in the trenches and making sure models are delivering the kind of impact they're looking for.
For each initiative, you've got to clear a path from the start that will lead to the integration of a predictive model. This requires a socialization of buy-in: Line of business leaders and managers must agree to make a real change to operations. They must learn what a predictive model does for them and must be willing to put their faith in it.
They're typically viewed as technology projects when they should be seen as organizational ones.
Then, even after this informed and detailed buy-in, business-side leaders and line-of-business managers must continue to remain involved with each stage of the ML project, including preparing the data, generating the model, deploying it, and measuring its performance.
Financial services has a long track record of successfully deploying this technology with credit scoring and fraud detection. They've carefully defined their goals and changed their internal processes. An ML project may simply be an incremental update to things they're already doing.
In logistics, UPS uses ML to predict where packages will need to be delivered tomorrow so they can start loading trucks in an optimal way the night before. Big tech companies like Google and Facebook had language translation models for years, but when they introduced deep learning algorithms, the translation quality improved overnight. They could do this because they already had much of the technology in place. It wasn't a huge change to operations, as it would be for companies whose ML practices are less mature.
Where you'll find the greatest potential to improve efficiency is in large-scale, legacy enterprise systems that have only just started to experiment with ML. That's where change management needs to play a huge role in making sure these projects ultimately get deployed.
Attachments
- Original Link
- Original Document
- Permalink
Disclaimer
ServiceNow Inc. published this content on 22 April 2022 and is solely responsible for the information contained therein. Distributed by Public, unedited and unaltered, on 25 April 2022 23:19:06 UTC.
