Join top executives in San Francisco July 11-12 to hear how leaders are integrating and optimizing AI investments for success. Learn more
Machine learning (ML) observability platform. Aporia today announced a strategic partnership with Databricks. According to the companies, the collaboration aims to empower customers using Databricks’ Lakehouse platform, AI capabilities and MLflow offerings by providing them with advanced monitoring capabilities for their ML models.
Organizations can now monitor their ML models in real time by deploying Aporia’s new ML observability platform directly on Databricks, eliminating the need to duplicate data from their lake house or any other data source.
Additionally, integration with Databricks simplifies the monitoring process, according to companies, by enabling analysis of billions of predictions without the need to sample data, make production code changes, or incur hidden storage fees.
“This means that tracking billions of predictions, visualizing and explaining ML models in production becomes simple,” Aporia CEO Liran Hason told VentureBeat. “Aporia supports all use cases and model types, providing flexibility for ML teams to tailor the platform to their specific needs.”
Real-time tracking, customization
The new offering enables monitoring of anomalies such as drift, bias, degradation and data integrity issues and triggers real-time alerts across popular communication channels, ensuring timely notifications.
The platform also provides real-time customizable dashboards and metrics, allowing each ML stakeholder to prioritize their key areas of interest and translate data science metrics into tangible business impact.
This is critical in areas such as loans, hiring and healthcare, Hason said, and promotes a fair and transparent landscape in automated decisions.
“Organizations would now be able to manage all ML models in one hub, regardless of implementation,” said Hason. “This centralization improves collaboration, facilitates communication, and simplifies model management, driving continuous learning and team workflow efficiency.”
Simplify data monitoring with ML Observability
Organizations have traditionally encountered difficulties when monitoring large volumes of data, often requiring duplication of data from their data lake to their monitoring platform. However, Hason said, this approach leads to inaccuracies, overlooked issues, drift, false positives, and difficulties in ensuring fairness and tracking bias.
The new integration with Databricks addresses these pain points by enabling organizations to quickly monitor all of their ML models on Databricks, in minutes.
Additionally, the integration maximizes the benefits of investments in existing databases, even for use cases that involve processing large volumes of predictions, such as recommender systems, search ranking models, fraud detection models, and demand forecasting.
“There’s no need to duplicate the data in a separate monitoring environment,” Hason explained. “This ensures a single source of truth derived directly from your data lake, simplifying data management and accelerating insights-to-action. The integration improves the effectiveness of ML model monitoring and provides flexibility and freedom for organizations to make the most of their ML potential and existing data infrastructure.”
Numerous use cases
The company said the new ML observability platform will support many use cases, including improving recommender systems through real-time performance monitoring.
Organizations can leverage Aporia to improve their search ranking algorithms, gain click-through rate insights, and improve search results. In addition, Aporia’s real-time monitoring helps detect and prevent fraudulent activity, strengthening security and fostering customer trust.
Additionally, the platform ensures accurate forecasting in supply chain management and retail by monitoring demand forecasting models, allowing teams to optimize their response to deviations from forecasted demand. The platform’s observability capabilities will also help financial institutions monitor credit risk patterns, ensuring accurate and unbiased credit ratings and identifying potential biases.
The Databricks Delta Connector establishes a connection between Aporia and an organization’s Databricks Delta Lake, linking training and inference datasets to Aporia, Hason explained.
The platform excels at monitoring large-scale data by effortlessly handling billions of predictions without resorting to data sampling, Hason said. This ensures a complete and accurate assessment of model performance, which is especially beneficial for organizations dealing with substantial volumes of data.
“No critical insight goes unnoticed, ensuring thorough monitoring,” he added.
Unleash the power of data for informed decision making
Hason said the partnership will play a crucial role in driving the broader adoption of observability across the AI and ML landscape, as it addresses existing demand and fuels a deeper understanding and recognition of observability as a cornerstone in the industry. ‘AI is in the ML.
He said the combination of a robust observability platform and a scalable data platform offers a compelling proposition for organizations investing in AI and ML. Companies are developing a unified tool that improves observability at scale, enabling organizations to make informed decisions and optimize their AI initiatives.
“The partnership is specifically designed to provide a centralized, end-to-end, cost-effective solution that enables organizations to make confident data-driven decisions,” added Hason.
Organizations can monitor all production data in minutes, ensuring rapid time-to-value. This accelerated implementation quickly unlocks the benefits of your investment.
“These new capabilities can save organizations valuable resources that would otherwise be spent troubleshooting and correcting problems,” said Hason.
VentureBeat’s mission it is to be a digital city square for technical decision makers to gain insights into transformative business technology and transactions. Discover our Briefings.
