Internet Explorer is not supported by our website. For a more secure experience, please use Chrome, Safari, Firefox, or Edge.
Infrastructure Software
Dharmesh Thakker, Danel Dayan, Jason Mendel  |  September 28, 2021
Seeing is Believing – Our Investment in Arize, and Why Machine Learning Observability is the Key to Unlocking the Power of AI

For several years, much has been made of the use of artificial intelligence (AI) or machine learning (ML) to power a variety of software applications used by all types of workers in all types of industries—from software developers at tech firms to doctors to construction managers to sales reps. Indeed, a recent Gartner survey showed that 92% of organizations polled plan to deploy AI in significant projects in the next three years.

But building useful AI technology isn’t as easy as creating traditional software for a host of reasons including the massive data requirements necessary in order to build robust models. So, a new crop of startups is trying to bridge that development gap through a new framework called “machine learning operations”, or MLOps for short. Today we’re thrilled to announce our investment in one of those startups, Arize AI*.

Arize, whose technology focuses on model observability, is tackling the huge problem of how organizations can continuously monitor and enhance ML-model performance—which, if done right, can help accelerate and deploy machine learning at greater scale across industries.

The broad category of MLOps technology is derived from core DevOps principles; MLOps ensures repeatable, reusable and consistent practices across the ML development workflow, where each step in the model-creation and deployment journey is broken down into its components so the upstream data is used properly and effectively.

There has been a flood of investment into the data-preparation, model-building and model-deployment pieces of the ML pipeline, and we’ve seen tremendous success in these areas from companies such as Databricks*, Dataiku*, Paperspace*, Scale AI and DataRobot, and open-source projects such as PyTorch and TensorFlow, which have emerged to address these pre-production pieces of the ML workflow.

However, building and shipping models is only the beginning of the AI journey. Once models are deployed into production, a common problem plagues every ML team: monitoring. Specifically, ML practitioners have historically lacked the necessary observability tooling to monitor the health and performance of AI systems, which, left to their own devices, are subject to a host of potential problems that can lead to costly model failures and erroneous model predictions.

Covid-19 has further exposed the need for better ML observability as models that were trained on normal, pre-Covid behavior are failing to perform in the post-pandemic world where the concept of “normal” has suddenly changed. Streaming-media companies, for example, have experienced recommendation-model inaccuracies (“if you watched this, you might like this”) from a rapid influx of new users. Many retailers can’t accurately predict order volumes due to Covid-induced order spikes, say for hand sanitizer or canned food. It all highlights how critical it is for organizations to have the proper ML observability tools to understand when and how model inputs are changing, and to catch and resolve issues before they impact customers.

Further, without ML observability, there’s a fundamental disconnect between the model’s performance and the data science and ML engineering teams responsible for the model’s integrity over time. From our conversations with several practitioners across G2000 and tech-forward organizations, we have seen that ML observability is a huge priority that needed to be solved yesterday, and is paramount to the operationalization of any models in production.

To that end, we’re excited to be partnering with Arize, whose observability platform provides real-time monitoring and explainability to help users understand how their ML models are performing in production–and then, take immediate action on any performance issues. By connecting datasets across training, validation and production environments, Arize enables ML teams to quickly detect when and where issues arise; deeply troubleshoot the cause behind any issues that emerge; and ultimately improve the model’s performance.

Companies like Sumo Logic*, Splunk*, Datadog*, New Relic, Elastic and Dynatrace have collectively amassed more than $100 billion in market capitalization, per Capital IQ, by solving the observability challenges that persist across the traditional software-development toolchain. While just as critical as this type of traditional, IT observability, ML observability is even more complex, given the interdependence of data, systems, features and code, which all impact the model’s performance. We think there is a large market opportunity for ML-observability tooling to provide a deep understanding of model performance as companies continue to invest significant resources into operationalizing AI.

Arize is led by seasoned practitioners, including Co-Founder/CEO Jason Lopatecki and Co-Founder/Chief Product Officer Aparna Dhinakaran, who have first-hand experience dealing with the pain points of deploying ML models into production, allowing them to balance creating a deeply technical product and delivering out-of-the-box usability. Arize’s founding executive and engineering teams cut their teeth at tech giants including Uber, Google, Apple, Slack, Adobe and PagerDuty.

We are excited to partner with Arize as the company brings monitoring and observability to machine learning and helps enterprises derive more value from their AI investments. We look forward to this next chapter of growth.

This material is provided for informational purposes, and it is not, and may not be relied on in any manner as, legal, tax or investment advice or as an offer to sell or a solicitation of an offer to buy an interest in any fund or investment vehicle managed by Battery Ventures or any other Battery entity. 

The information and data are as of the publication date unless otherwise noted.

Content obtained from third-party sources, although believed to be reliable, has not been independently verified as to its accuracy or completeness and cannot be guaranteed. Battery Ventures has no obligation to update, modify or amend the content of this post nor notify its readers in the event that any information, opinion, projection, forecast or estimate included, changes or subsequently becomes inaccurate.

The information above may contain projections or other forward-looking statements regarding future events or expectations. Predictions, opinions and other information discussed in this video are subject to change continually and without notice of any kind and may no longer be true after the date indicated. Battery Ventures assumes no duty to and does not undertake to update forward-looking statements.

*Denotes a Battery portfolio company. For a full list of all Battery investments, please click here.

Back To Blog