Last year, Databricks made headlines by acquiring MosaicML for $1.3 billion. The rebranded platform, now known as Mosaic AI, has since become a cornerstone of Databricks’ AI suite. At the recent Data + AI Summit, the company unveiled a range of new features designed to enhance enterprise AI capabilities. Ahead of these significant announcements, I had the chance to speak with Databricks co-founders, CEO Ali Ghodsi and CTO Matei Zaharia.
Introducing Five New Mosaic AI Tools
At the summit, Databricks introduced five innovative tools under the Mosaic AI umbrella: Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training, and Mosaic AI Gateway.
“It’s been an incredible year filled with advancements in generative AI. The key questions remain consistent: how do we improve the quality and reliability of models, ensure cost-efficiency despite significant price variances, and maintain data privacy?” said Ghodsi.
These new offerings are specifically designed to address these essential concerns for Databricks’ clientele.
Enhancing Large Language Model (LLM) Deployment
Zaharia highlighted that enterprises deploying large language models (LLMs) often rely on systems with multiple components, necessitating several model calls and the use of external tools for functions such as retrieval augmented generation (RAG). These multifaceted systems boost LLM application speed, reduce costs with model-specific queries, and enhance output reliability through integration with proprietary data.
“We believe that this modular approach represents the future of critical, high-impact AI applications. Modular systems allow engineers to control all aspects and make targeted improvements,” Zaharia explained. “Our ongoing research aims to develop the optimal systems for specific tasks, facilitating developer interaction and traceability.”
New Services for System Building and Evaluation
To support the construction of these modular systems, Databricks has launched the Mosaic AI Agent Framework and the Mosaic AI Tools Catalog. The AI Agent Framework integrates serverless vector search functionality, which recently became available, enabling developers to create RAG-based applications.
Ghodsi and Zaharia emphasized the hybrid nature of the Databricks vector search system, combining classic keyword search with embedding search. Integrated with Databricks’ data lake, the system ensures data synchronization and governance, including the Databricks Unity Catalog governance layer, to prevent personal information leaks.
Databricks is also extending the Unity Catalog to manage which AI tools LLMs can use during response generation, enhancing service discoverability across organizations.
Empowering Developers and Ensuring Model Quality
Developers can now utilize these tools to construct customized agents by chaining together models and functions via Langchain or LlamaIndex. As Zaharia noted, many Databricks customers are already leveraging these tools effectively.
For application evaluation, Databricks is introducing the Mosaic AI Agent Evaluation tool. This AI-assisted tool uses LLM-based judges to test AI performance in production environments while enabling quick feedback and initial dataset labeling.
“Virtually every customer needs some form of internal data labeling. They might start with 100 to 500 answers, which can then be fed into the LLM judges,” Ghodsi elaborated.
To further improve AI outcomes, Databricks now offers the Mosaic AI Model Training service. This service allows organizations to fine-tune models with their private data, enhancing performance for specific tasks.
Comprehensive AI Management with Mosaic AI Gateway
The final addition, Mosaic AI Gateway, serves as a unified interface for querying, managing, and deploying both open-source and proprietary models. This tool helps organizations query LLMs securely, using a centralized credential store, and includes features like setting rate limits for vendors to control costs and tracking usage for debugging.
Ghodsi highlighted that these features are a direct response to evolving customer needs. “We’ve observed a market shift recently. While open-source models are praised, many turn to Open AI. As sophistication increases, our customers adopt new tools to address emerging challenges and opportunities.”
These enhancements underscore Databricks’ commitment to equipping enterprises with robust, scalable AI solutions through Mosaic AI.
