Splunk Unveils Splunk AI to Facilitate Security and Observability Through Generative AI

During Splunk’s .conf23 event, the company announced Splunk AI, a suite of AI-powered solutions designed to enhance its unified security and observability platform. According to the company, the latest development combines automation with human experiences to enable organizations to improve their detection, investigation and response capabilities while maintaining control over the implementation of AI.

The new Splunk AI Assistant uses generative artificial intelligence to provide users with an interactive chat experience using natural language. Users can build Splunk Processing Language (SPL) queries through this interface, thereby broadening their understanding of the platform.

Through the AI ​​Assistant, Splunk aims to optimize time to value and increase accessibility to SPL, democratizing an organization’s access to valuable data insights.

Splunk said the AI ​​will enable SecOps, ITOps and engineering teams to automate data mining, anomaly detection and risk assessment. so they can focus on more strategic activities and reduce errors.

“As a company, we have been deliberate in ensuring that our Splunk AI innovations combine automation with human-in-the-loop experiences, so organizations can strengthen human decision-making with threat response by increasing speed and effectiveness, but not by replacing human decisions,” Splunk CTO Min Wang told VentureBeat. “Both of our core and integrated AI offerings within Splunk AI provide recommendations on large and rich information sets to improve and accelerate human decision-making around detection, investigation, and response.”

The model is integrated with domain-specific large language models (LLM) and ML algorithms, leveraging security and observability data to increase productivity and cost efficiency. The company emphasized its commitment to openness and extensibility, as it allows organizations to integrate their own AI models or third-party tools.

“What differentiates Splunk’s AI-powered offerings is that they optimize large domain-specific language models and ML algorithms based on security and observability data,” Wang told VentureBeat. “These domain-specific insights will provide SecOps, ITOps and engineering teams with relevant data to automatically detect anomalies and then prioritize their attention where it is most needed based on intelligent risk assessment, minimizing repetitive processes and ‘human error”.

Easing security and IT workloads through artificial intelligence

Splunk says that as the technology infrastructure becomes more complex and distributed, and with a continuing shortage of talent, organizations need tools that allow them to act quickly and efficiently without exhausting their teams.

“With Splunk AI, we want to help streamline the work of SecOps, ITOps and engineering, so they can focus on more strategic work… (and) act faster and more accurately to ensure their systems remain resilient” said Wang of Splunk.

Splunk’s new AI-powered capabilities aim to improve the speed and accuracy of alerts while strengthening digital resiliency. According to the company, its anomaly detection app simplifies and automates the entire anomaly detection operational workflow.

Meanwhile, IT Service Intelligence 4.17 introduces outlier exclusion for Adaptive Threshold, which identifies and excludes outlier data points. Additionally, “ML-assisted thresholding” generates dynamic thresholds based on historical data and models, resulting in more accurate alerts.

“ML-assisted thresholding uses historical data and models to create dynamic thresholds with a single click. Thresholds that better reflect the anticipated workload on an hourly basis help ITOps and engineering teams reduce false positives and generate more accurate alerts on the health of an organization’s technology environment,” Wang explained.

In another development, the company unveiled foundational offerings powered by machine learning that ensure organizations have access to comprehensive information. The Splunk Machine Learning Toolkit (MLTK) 5.4 now provides guided access to ML technology, enabling users of all skill levels to take advantage of predictions and predictive analytics.

“MLTK can be deployed on (the) Splunk Enterprise or Cloud platform to extend the platform with techniques such as outlier and anomaly detection, predictive analytics and clustering, to filter noise and address common ML use cases,” he said Wang.

Wang said the latest version of MLTK allows users to easily upload their pre-trained models to MLTK through a user-friendly interface.

Once the model is in Splunk, users can seamlessly apply it to their Splunk data without disrupting existing workflows. This feature expands the applicability of MLTK and ML-SPL to include models trained using methods other than MLTK.

Emphasize data science for better discovery and analysis

According to Wang, domain specificity is crucial for models. He stressed the importance of fine-tuning the templates specifically for their use cases and having them built by industry experts. While generic models of large languages ​​(LLMs) can serve as a starting point, you said the most effective models are those tailored to specific domains.

Wang pointed out that while generative AI is valuable for learning curves and generating new information, deep learning tools may be better suited for incorporating complex, purpose-built anomaly detection algorithms into security offerings.

“As experts in security and observability, I believe we have the best domain-specific insights derived from real-world experience from our development team, go-to-market team, and customers,” he said.

To facilitate this transition, Splunk has introduced Splunk App for Data Science and Deep Learning (DSDL) 5.1. This MLTK extension enhances the integration of advanced custom machine learning and deep learning systems with the Splunk ecosystem, thus strengthening its capabilities.

“The DSDL extends MLTK with pre-built Docker containers for additional machine learning libraries. Data scientists and machine learning or deep learning engineers can use DSDL to leverage GPU computing for compute-intensive training tasks and flexibly deploy models on CPU- or GPU-enabled containers,” Wang explained. This offering is specifically for our customers who store their data in Splunk environments and need tools to incorporate powerful ML algorithms trained on their data for their unique purposes.”

DSDL 5.1 also introduces two new AI assistants that will allow customers to use LLM to build and train models specific to their domain. These assistants will focus specifically on text summarization and text classification applications.

Wang said AI/ML and analytics are key to improving anomaly detection and alerting accuracy. These technologies reduce false positives and tailor thresholds based on unique customer data patterns, resulting in more effective alerts.

Along the same lines, the company’s new Splunk anomaly detection app uses machine learning to automate anomaly detection in your environment. It also offers consistent health diagnostics.

“The app provides an end-to-end operations workflow so organizations can create and run consistent anomaly detection jobs, view SPL queries, and create alerts. This leads to more accurate general alerts,” Wang said.