Foundation Models: The Multi-Billion Dollar Bet on AI's Future
Introduction
At the forefront of the excitement surrounding AI over the past few years are what experts call "foundation models." These are advanced AI systems trained on vast amounts of data that can perform a wide range of tasks. You may have heard of some popular examples like GPT-4o, Claude 3.5, Llama 3.1, or Gemini 1.5. These AI systems can write, code, create art, and chat like humans. Major tech companies are investing large sums in building the underlying infrastructure and know-how to operate these new systems.
Meta (Facebook) plans to spend up to $40 billion this year on AI
Alphabet (Google) increased AI spending by 91% to $12 billion in just one quarter
Microsoft boosted AI investment by 79% to $14 billion in a quarter, aiming for $50 billion this year
Amazon invested $14.9 billion in AI-related infrastructure, with plans to spend even more
Companies are betting on these systems’ potential to revolutionize business operations and open up new revenue streams. These massive investments show how seriously tech leaders are taking AI's potential. But, some analysts are starting to question: Will these billion-dollar bets lead to the game-changing, large payout breakthroughs or are there risks of overspending without clear returns?
Before answering these questions, let's review (a) what these models are, (b) their potential applications and economic impact, and (c) some of the challenges in applying them at the scale current expectations suggest.
What Are Foundation Models?
Foundation models are advanced AI systems trained on vast amounts of diverse data, enabling them to perform various tasks. They can handle many different jobs, from understanding and generating text to recognizing images.
In the news, you might hear terms like Large Language Models (LLMs) and Generative AI (GenAI), which are specific types of foundation models. LLMs, such as ChatGPT, are specialized in language tasks, while GenAI models can create various types of content. For example, Midjourney generates images, and Sora creates videos, all from text descriptions.
One of the key technologies behind foundation models is the transformer architecture. This technology helps the AI understand context and connections in data, making it much smarter and more versatile than older systems. Essentially, transformers allow the AI to grasp the relationships between different pieces of information, much like how humans understand the context in a conversation or a story. This ability to understand context is what allows foundation models to be so adaptable and powerful across various tasks.
Foundation models differ significantly from traditional AI systems (like multiple regression models used in high school statistics). Traditional AIs are trained on specific datasets for particular tasks, such as recognizing handwritten digits or predicting house prices. They're like toasters - great at one specific job, but inflexible. Just as a toaster can only toast bread and can't suddenly start grilling meat or baking cakes, these AIs can't adapt to tasks outside their training. In contrast, foundation models are more like skilled chefs - versatile and adaptable. A chef can create a wide variety of dishes, adapt recipes on the fly, and even invent new culinary creations based on their broad knowledge. Similarly, foundation models can apply their vast knowledge to diverse tasks and easily learn new skills.
But this power does not come without a hefty resource requirement, they demand massive datasets for training, lots of storage and substantial computing power. Developing and maintaining them also requires teams of highly specialized AI researchers and engineers. These resource demands contribute to the high costs associated with foundation models and present challenges for widespread adoption, especially for smaller organizations or in resource-constrained environments.
Real-world applications of foundation models1
These new systems can now handle complex tasks that previously required human expertise, with the hope that these systems will help businesses automate operations at scale or create new subscription services, examples of use cases are:
Content Creation and Marketing: Automating content creation, ad copywriting, and personalized email campaigns.
Customer Support: Enabling natural, context-aware chatbots and virtual assistants for improved customer service.
Language Services: Facilitating seamless translation and localization (adapt content for specific regions, ensuring cultural relevance) for global businesses.
Summarization Services: Summarizing lengthy documents and extracting specific data, enhancing productivity and decision-making.
Customer Intelligence: Analyzing sentiment and identifying trends from vast amounts of customer feedback and market data.
Legal and Compliance: Streamlining contract analysis and regulatory compliance processes.
Financial Analysis & Decision Support: Distilling complex financial reports and news for actionable insights.
Education: Providing automated assistance, tutoring and generating educational content to help students learn at a personalized level.
Economic Impact Predictions
Widespread adoption of foundation models has the potential to revolutionize our work lives as profoundly as the internet, computers, and smartphones did, with potentially major financial implications. Some of the expert predictions on the size of this benefit are:
McKinsey Global Institute (2023) predicts AI could create $2.6 trillion to $4.4 trillion annual addition to the global economy
ARK Invest, a top tech hedge fund, believes the adoption of foundation models could create $80 trillion in enterprise value by 2030.
These changes aren't just about profits; they're reshaping the job market too. The World Economic Forum's Future of Jobs Report 2023 predicts that 23% of global jobs will change in the next five years due to industry transformation, including through artificial intelligence and other technologies. Suggesting a significant shift in the roles of future and the skills required to do them.
Word of Caution: There Are No Overnight Successes
Experts like Erik Brynjolfsson from Stanford remind us that big technological changes often take decades to make a difference. Electricity and computers, took decades to significantly impact productivity. Right now, we're still in the “figuring it out stage” with AI. The current state of AI adoption reflects this reality: A 2023 Boston Consulting Group survey found that while 94% of companies are experimenting with generative AI, only 6% are using it at scale. Companies struggle to fully adopt AI due to messy data, clunky software integration, fears about AI making costly mistakes, and the challenge of teaching employees new tech skills while overhauling familiar business practices.
The AI revolution is a marathon, not a sprint. While foundation models have incredible potential, widespread adoption will take time. We need to balance excitement with realism, focusing on practical AI applications that can truly make a difference. The AI bet has been placed, but the payoff requires patience and wise leadership. Now, it's our move to play the hand skillfully and shape the outcome of this technological gamble.
Those are my Thoughts From the DataFront
Max
A September report by Lakera’s Deval Shah provides a list of eleven real-world applications