Learn what the Databricks acquisition means for companies looking into public or private generative AI foundation models.
MosaicML will join the Databricks family in a $1.3 billion deal and provide its “factory” for building proprietary generative artificial intelligence models, Databricks announced on Monday. Companies can use AI like these to ease fears of intellectual property breaches.
The combination of Databricks’ data management technology and MosaicML’s ability to build AI models will let companies create their own large language platforms instead of relying on public generative AI such as OpenAI’s ChatGPT.
MosaicML has created two generative AI foundation models: MPT-7 (with 6.7 billion parameters) and MPT-13 (with 29.9 billion parameters). The MPT foundation models will join Databricks’ own open-source LLMs: Dolly 1 and 2.
Jump to:
MosaicML was the right choice for the Databricks acquisition because it has the “easiest factory on the market to use,” Databricks CEO and co-founder Ali Ghodsi said at the Databricks + AI summit on Tuesday.
He also cited a similar, competitive company culture as a reason why MosaicML was a good fit.
The acquisition is still going through regulatory approval; the deal is expected to close by the end of July. Databricks will have more information on how MosaicML’s AI training and inference products will integrate with Databricks software after that process has completed, Ghodsi said.
Databricks primarily provides data storage and data management software for enterprise organizations, as well as handles data platform migration and data analytics. Databricks has partnerships with AWS and other large enterprise software and software-as-a-service providers.
Ghodsi pointed out that his company will use MosaicML’s resources to provide “factories” where customers can build and train LLMs to their own specifications. This means companies won’t have to shell out for application programming interface connections or share proprietary data with anyone else who uses the model; the latter has become a concern for companies using ChatGPT or Google Bard. Databricks customers will be able to choose between the Dolly and MPT families or build a custom generative AI on one of the existing models.
SEE: Tips on how to decide whether a public or private generative AI model is right for your organization (TechRepublic)
Whether to use closed source or open-source AI foundation models is the battle on everyone’s mind today, Ghodsi said. Databricks is firmly on the side of open source.
“We think that it’s better for everybody if there’s open research on understanding these models,” Ghodsi said during a Q&A session at the summit. “It’s important we understand their strengths, their weaknesses, their biases and so on.
“But we also think that, most importantly, companies want to own their own model … They don’t want to use just one model that someone has provided, because it’s intellectual property. And it’s competitive.”
Customers want to control their own IP and keep their data locked down, Ghodsi said.
Junaid Saiyed, chief technology officer of data management and analytics software company Alation, also finds customers asking about generative AI. However, it’s important for organizations to know the data they are feeding the training model is good, he said in an email to TechRepublic.
“The proliferation of data sources and increasing data volumes have made it more difficult than ever for people to search for and discover the trusted, governed data they need to train their AI models,” Saiyed said. “To be really effective, generative models must be fine-tuned on domain-specific data catalogs, and humans should review their output.”
Umesh Sachdev, co-founder and chief executive officer of conversational AI and automation company Uniphore, recommends enterprise leaders ask themselves the following questions when deciding whether to build their own AI on a foundation model like MosaicML’s or to use public AI like the GPT series:
“For most organizations, they have specialized tasks that they want to do … and for that, we want them to be able to train and tune specific models,” Ghodsi said at the Databricks + AI summit.
Enterprise customers need a certain threshold of technical skill to build generative AI, Ghodsi said. He anticipates that MosaicML can fill a need for an easier way to build and train AI technology.
“Hopefully, eventually, we’ll make it something you can do with a few clicks,” Ghodsi said at the summit.
“This technology (generative AI) is in its nascency, and a lot needs to be uncovered about data sovereignty, scalability and cost,” said Sachdev in an email to TechRepublic. “Companies are moving fast to make announcements and decisions, but like most big tech waves, the opportunities will unfold in the second or third wave of development.”
“This AI transformation is revealing to business and technology leaders what the true state of their data environment is,” Saiyed said. “Organizations with a data intelligence platform and federated data governance will be able to leverage the power of GenAI before those that are only now investing in modernizing [their] data management strategy.”
Competition in the area of AI training is fierce; MosaicML competes with NVIDIA, OpenAI, Anthropic and Google. On Monday, NVIDIA announced a partnership with Snowflake to add the NVIDIA NeMo LLM development platform and NVIDIA GPU-accelerated computing to the Snowflake Data Cloud.
Four other major updates came out of the Databricks + AI summit:
