This content is part of the Essential Guide: Guide to using advanced analytics and AI in business applications

Essential Guide

Browse Sections

Automating machine learning puts analytical models on autopilot

With data scientists in short supply and the value of machine learning models growing ever more evident, software vendors are increasingly looking to automate machine learning.

If you've ever tried to write machine learning models, you know how complicated they can be. But that may not be a big concern in the not-so-distant future.

That's because software vendors are increasingly inserting machine learning functionality deep within their applications, automating machine learning processes for users who lack data science chops.

"I never saw this kind of thing being a part of my career," said Julia Kurnia, founder of Zidisha Inc., a microlending service based in Sterling, Va. Zidisha uses software from DataRobot Inc. to run machine learning models that assess the credit-worthiness of applicants and score them for fraud risk.

Kurnia, who has degrees in political science and economics, said she picked up some basic SQL coding skills over the years, but found data science a tougher nut to crack. Her nonprofit, which started as a two-person operation, had few resources to hire a data scientist. The organization relied on volunteer efforts for a while, but it was hard to develop a consistent approach to its machine learning needs.

No machine learning experience needed

Now, Kurnia is able to use machine learning models that are generated automatically by the DataRobot software. She uploads data generated during the application process, along with data from Zidisha's website, and tells the cloud-based software what she wants to learn from it. The software automates model selection and runs the optimal machine learning model on the data set. Kurnia doesn't have to worry about feature engineering or any of the more complex stages of machine learning.

Kurnia said that since she started using the software, she has been able to reduce loan defaults by about 5%.

And she thinks more small organizations will likewise adopt machine learning and predictive analytics because the barriers to entry are getting lower as software automates more of the process. This means that organizations don't have to shell out big bucks to attract data scientists skilled in building neural networks or make massive investments in sophisticated software.

"I would expect more small companies like ours to start using data science earlier and earlier," Kurnia said. "It's just a matter of time."

Beyond the basics on automation

More than basic machine learning models are becoming automated. Today, software exists that can do more complicated tasks, like natural language processing and generation that does not require coding. In the future, even artificial intelligence could be automated for enterprises.

For example, San Antonio-based USAA is using a natural language tool from Narrative Science that runs machine learning algorithms to automatically generate verbal descriptions of metrics included in business intelligence reports. Luke Horgan, director of digital channel analytics at the insurance company, said this enables his team to answer a broader array of questions than they were previously capable of tackling.

How to prepare for machine learning projects

Before implementing the tool, users of USAA's reports would inevitably have questions about some metric or score included in it. Horgan and his team would spend a significant amount of time responding to these questions. But now, the software's automatically generated metric descriptions pre-empt a lot of the questions, and it's all done at scale without any need for coding. The software works by enabling Horgan's team to define a few preferences and then it builds the predictive models.

"Someone like myself without a lot of knowledge can do something really quickly," Horgan said.

The trend of automating machine learning via embedded tools is playing out broadly across the software industry. For example, data visualization vendor Tableau Software plans to introduce a data preparation tool called Project Maestro that uses machine learning to automate many steps of preparing data for exploration.

In an interview, IDC analyst David Schubmehl said vendors including Microsoft, Oracle and Salesforce have also made recent announcements that they're embedding machine learning under the hood of their software.

Machine learning models for all comers

Soon, more and more enterprises may be using machine learning without even realizing it, according to Schubmehl. "The idea is that using machine learning allows people to put more of the software tools on autopilot," he said. "Think of it as the next wave or generation of enterprise software."

The trend makes sense, Schubmehl said, because the benefits of machine learning are so obvious, yet the highly educated data scientists needed to develop neural network and machine learning applications are so scarce. Every enterprise stands to benefit from becoming more data-driven, yet developing capabilities internally is expensive and time-consuming.

"There is a shortage of data scientists, so that makes it more challenging," he said. "This idea of low-code or no-code AI is really a great capability. Automated machine learning helping to drive the enterprise is really where our research is pointing to."

Still, that doesn't mean that automating machine learning automatically delivers a return on investment. Schubmehl described it as a hand-and-glove situation. In this analogy, data is the hand and software is the glove. If the data isn't there, the software can't accomplish much.

He said enterprises will have to invest in infrastructure to acquire, clean and stage data before they can benefit from software offering automated machine learning capabilities. This is the biggest barrier that organizations could stumble over as they look to implement the software.

That said, Schubmehl thinks most organizations will look before they make the leap to automated machine learning software.

"I think people do understand that these are data-driven algorithms," he said. "Some organizations don't really have a good handle on their data strategy, but more organizations are thinking about data strategy."

Next Steps

Learn about the hottest trends in machine learning

Discover how advanced analytics bolsters BI

Dig Deeper on Machine learning modeling