Announcing HackAPrompt 2.0: The World’s Largest AI Red-Teaming Hackathon

Two years ago, we announced HackAPrompt 1.0, a first-of-its-kind prompt-hacking competition. We asked participants to break chatbots and pay money to those who proved the most adept at it. Until then, no one had ever organized a hackathon dedicated exclusively to “prompt hacking” vulnerabilities in large language models (LLMs).

More than 3,000 participants joined us in injecting, leaking, and outright defeating the infamous sandwich defense 🥪. Collectively, they submitted over 600,000 malicious prompts uncovering serious vulnerabilities!

Today, we’re thrilled to announce HackAPrompt 2.0. This second iteration shifts the spotlight to general AI red teaming, attacking not just simple chatbots but also AI agents, robotics systems, and beyond.

HackAPrompt 2.0 is bigger, broader in scope, and now features five specialized tracks for participants of varying skill levels. With $500,000 in prizes, HackAPrompt 2.0 stands as the largest AI safety competition ever.

A Look Back at HackAPrompt 1.0

Sander Schulhoff, our CEO, first learned about prompt injections in 2022 from Riley Goodside, known as the world’s first “Staff Prompt Engineer” at Scale AI, who publicly highlighted and recognized the severity of such attacks. Prompt injection vulnerabilities quickly proved to be a significant security concern, with real-world examples surfacing at a rapid pace.

An Early Example of Prompt Injection

One notable incident involved a remote-work company, Remoteli.io, which ran a Twitter bot designed to respond positively to tweets about remote work. People soon realized they could insert special instructions into their tweets to inject hidden commands.

In one case, user Evelyn’s last line instructed the bot to threaten the president, exposing how easily LLM-based systems can be hijacked if not secured.

Industry-Wide Support

Recognizing the urgency of these vulnerabilities, we teamed up with OpenAI, ScaleAI, Hugging Face, and 10 other AI companies to launch HackAPrompt 1.0. The enthusiastic response proved that such red-teaming events are essential in exposing and mitigating emerging security threats in AI systems.

Why We’re Creating HackAPrompt 2.0

AI doesn’t stand still. after HackAPrompt 1.0, we’ve seen a rapid expansion toward autonomous AI (AI agents), capable of planning, reasoning, and adapting in real time, and multimodal AI, which processes various data types simultaneously (text, images, audio, etc.).

As we discussed in our 2025 AI trends newsletter, this year, we'll soon see AI applications that can handle complex, multi-step tasks on personal devices with real-time adaptability.

Concepts like embodied humanoid robots, generative AI integrated into military command systems, and advanced AI agents have moved beyond science fiction and are making their way into everyday use. Yet these systems, like their simpler chatbot predecessors, remain susceptible to hacking and exploitation.

Real-World Impact

AI systems are no longer confined to specialized applications. As AI becomes more autonomous and deeply integrated into sectors like logistics, finance, healthcare, and defense, the stakes rise significantly. With the ability to autonomously shop, plan military operations, and manage tasks on personal devices, these systems are taking on increasingly complex and impactful roles. Yet, like their simpler predecessors, they remain vulnerable to hacking and exploitation. New capabilities inevitably introduce new risks.

Our goal is to research the next generation of AI threats, and we need you to help by stress-testing these systems. Let’s discover the flaws now, so we can protect against them in the real world.

What’s New in HackAPrompt 2.0?

1. Expanded scope: Beyond basic chatbots, we’re now exploring full-spectrum AI red teaming, including LLMs connected to external tools and advanced agent-based systems.
2. Five specialized tracks: Ranging from beginner-friendly (Classic track) to advanced (Future Attacks), allowing participants of all skill levels to join.
3. Larger prize pool: $500,000 in total prizes, making it the largest AI safety competition to date.
4. Longer duration: Spread out over 2 months, giving you more time to experiment, collaborate, and hone your strategies.
5. Broader partnerships: We’re continuing and expanding our partnerships with top AI companies to ensure diverse and challenging vulnerabilities across multiple domains.

Tracks Overview

HackAPrompt 2.0 is divided into five key tracks (with two upcoming) to accommodate different experience levels and interests.

Track 1: Classic (Beginner)

This track covers the foundational challenges typically associated with GenAI red teaming—everything from prompt injections to misinformation and hate speech. Even though these are “classic” vulnerabilities, they still pose a serious threat when exploited at scale.

Track 2: Agents (Intermediate)

Moves beyond prompt-based attacks to scenarios where LLMs have tool access, making them “agentic.” This could mean the AI physically interacts with the world (e.g., in a robot) or programmatically performs tasks (e.g., writing code, conducting web searches).

Track 3: Future Attacks (Advanced)

Focuses on next-generation threats emerging from increasingly capable GenAI systems. Attack vectors here are more speculative but increasingly plausible as AI technology rapidly advances.

(Levels 4 and 5 will be announced soon. Watch our channels for updates!)

Summary Table

Track	Skill Level	Focus Areas	Example Attack Types
Track 1: Classic	Beginner	Known “GenAI Red Teaming” harms Further study of fundamental LLM vulnerabilities	CBRNE: Misuse of LLMs for generating harmful chemical, biological, radiological, nuclear, or explosive content Misinformation: Large-scale manipulation of public perception Harmful language and hate speech
Track 2: Agents	Intermediate	Focus on “agentic” LLMs with access to external tools Real-world impacts of LLMs beyond mere text output	Real-world embodied LLMs: Humanoid robots or systems tricked into harmful physical actions Software development agents: Manipulated to create malicious code Internet search agents: Exploited to gather harmful information or actions
Track 3: Future Attacks	Advanced	Cutting-edge threats difficult to foresee Vectors emerging from rapidly evolving GenAI capabilities	AI Virus: Intelligent, adaptive malware that leverages LLM APIs to evolve in real time GenAI-Integrated Command & Control Systems: Vulnerabilities in GenAI-driven C2 infrastructure, potentially basing decisions on untrusted data
Track 4: N/A	Coming Soon	(Details to be revealed)	—
Track 5: N/A	Coming Soon	(Details to be revealed)	—

How to Sign Up

Ready to be part of the world’s largest AI red-teaming competition? Here’s what you need to do:

1. Join the waitlist: Sign up here.
2. Mark important dates: We’ll email you about the official start date, track details, and deadlines.
3. Prepare your toolkit: Familiarize yourself with prompt injection tactics, agent frameworks, and current or emerging AI threats.

Questions?

Feel free to reach out at [email protected] or follow us on social media for the latest updates.

Useful Links

• HackAPrompt 2.0 Website
• Sign up form
• Prompt Hacking Docs

Valeriia Kuka

Valeriia Kuka, Head of Content at Learn Prompting, is passionate about making AI and ML accessible. Valeriia previously grew a 60K+ follower AI-focused social media account, earning reposts from Stanford NLP, Amazon Research, Hugging Face, and AI researchers. She has also worked with AI/ML newsletters and global communities with 100K+ members and authored clear and concise explainers and historical articles.