Key Takeaways from State of AI Report 2024 and 2025 Predictions

October 24th, 2024 by Valeriia Kuka

Nathan Benaich and Air Street Capital just released their State of AI Report 2024, a massive 200+ slide presentation covering everything from cutting-edge research to industry trends, politics, safety, and predictions for 2025. We’ve done the heavy lifting and summarized the key takeaways to save you time.

We cover:

Section	Subtopics
2023 Predictions	What Hit and What Missed
2025 Predictions	- $10B+ Sovereign Investment in U.S. AI Lab - No-code App Success - Data Collection Reforms - Softer EU AI Act - Open-source Model Surpasses OpenAI’s o1 - NVIDIA’s Market Dominance Continues - Humanoid Robot Investment Declines - Apple’s On-Device AI Success - AI-Generated Research Paper - AI-Centered Video Game Breakthrough
Key Research Trends	- Frontier AI Labs - Multimodal AI - OpenAI o1 - Meta’s Llama 3 - Challenges
Industry Highlights	- NVIDIA’s Reign - Generative AI Revenue - Skyrocketing AI Valuations
Political and Geopolitical Dynamics	- AI Regulation - China’s AI Surge - Sustainability Concerns
AI Safety	- AI Safety vs. Speed - State Safety Capacity
Technology Developments	- Smaller AI Models for Mobile - Quantization & Distillation

Review of 2023 Predictions: What Hit and What Missed

Here’s a quick review of the 2023 predictions and how they panned out:

Outcome	Prediction	Details
Worked	Hollywood-grade production using Generative AI for VFX	Generative AI used in Netflix and HBO productions, though many were poorly executed.
Worked	Microsoft/OpenAI partnership investigated by regulators	Investigations launched by FTC and UK’s CMA.
Worked	Limited progress on global AI governance	High-level voluntary commitments were made, but no binding actions.
Worked	AI-generated song breaking into Billboard Hot 100/Spotify Top Hits	AI-generated songs charted, including #27 in Germany, with previous success on Billboard.
Worked	AI company building/acquiring inference-focused chip company	OpenAI’s Sam Altman raising funds, while major tech companies work on their own AI silicon.
Partially	Generative AI media company investigated during 2024 U.S. elections	No investigation yet, but the election cycle isn’t over.
Partially	Tech IPO markets unthaw with AI companies listing	Public companies saw gains, but private companies remain cautious; Cerebras has filed for IPO.
Failed	Self-improving AI agents crushing SOTA in complex environments	No breakthroughs despite advancements in open-endedness and game performance.
Failed	Generative AI scaling with $1B+ investment	No reports of such a large investment yet, but expectations remain high.
Failed	Financial institutions launching GPU debt funds	No official GPU debt funds, though rumors of VC funds offering GPUs for equity exist.

2025 Predictions

1. $10B+ Sovereign Investment in U.S. AI Lab: A major investment from a sovereign state will trigger a national security review.

2. No-code App Success: An app or website created by someone without coding skills will go viral and break into the App Store Top-100.

3. Data Collection Reforms: After legal cases reach trial, frontier labs will begin making meaningful changes to data collection practices.

4. Softer EU AI Act: Initial implementation of the EU AI Act will be more lenient than expected, due to concerns about overregulation.

5. Open-Source Model Overtakes OpenAI o1: An open-source AI model will surpass OpenAI's o1 on reasoning benchmarks.

6. NVIDIA's Market Dominance Continues: Challengers will struggle to make a significant impact on NVIDIA’s market position.

7. Humanoid Investment Declines: Investment in humanoid robots will decrease as companies face difficulties achieving product-market fit.

8. Apple's On-Device AI Research: Strong results from Apple’s on-device AI efforts will drive momentum around personal AI assistants.

9. AI-Generated Research Paper: A research paper generated by an AI scientist will be accepted at a major machine learning conference or workshop.

10. AI-Driven Video Game Breakthrough: A video game centered around AI-generated content will gain mainstream attention.

Key Research Trends

• Frontier lab performance: AI research labs have achieved significant strides, with models like GPT-4o, Claude 3.5, and Gemini 1.5 converging in their performance. Reasoning and planning have emerged as key frontiers.
• Multimodal AI: Research now extends AI beyond language, delving into mathematics, biology, and physical sciences, marking a shift toward multimodal systems that integrate various forms of data, from text to biological structures.
• OpenAI’s o1: OpenAI’s latest models (like o1) have made notable advancements in reasoning, particularly in solving complex mathematical and scientific problems.
• Meta’s Llama 3: Llama 3 models close the gap between proprietary and open models, achieving significant performance on reasoning, math, and multilingual tasks.
• Challenges: Dataset contamination (test data leaking into training data) is an ongoing issue that could be inflating performance metrics across the field.

Industry Highlights

• NVIDIA’s Reign: NVIDIA remains the dominant player in AI hardware, though regulatory scrutiny is intensifying.
• Generative AI Revenue: AI companies are raking in billions, but sustainability questions loom, particularly around pricing models.
• Skyrocketing AI Valuations: AI companies hit a $9 trillion valuation, driven by a strong stock market and increased venture capital.

Political and Geopolitical Dynamics

• AI Regulation: While global AI governance efforts have stalled, the U.S. and EU are making headway with regional regulations, despite controversies.
• China’s AI Surge: Despite facing tech sanctions, Chinese AI models are catching up with U.S. counterparts.
• Sustainability Concerns: The massive compute demands of AI systems are raising red flags about energy consumption and environmental impact.

AI Safety

• AI Safety vs. Speed: Companies are shifting focus from safety concerns to accelerating AI development, despite risks like sophisticated jailbreaking.
• State Safety Capacity: Governments are setting up AI safety institutions, similar to the UK’s research institutes for critical infrastructure protection.

Technology Developments

• Smaller models for mobile: Research and development in reducing model sizes have progressed, with companies like Apple, Microsoft, and Hugging Face optimizing models to run on smartphones and edge devices.
• Quantization and distillation: These techniques allow AI models to operate efficiently with reduced precision, maintaining strong performance while lowering computational demands.

Conclusion

The 2024 State of AI Report gives us a glimpse of how rapidly the AI landscape is evolving, from research breakthroughs to major industry shifts. Whether it’s NVIDIA’s hardware dominance or the rise of no-code apps, it’s clear AI is reshaping the future in ways we’re only beginning to grasp.