Chapter 5.1 - Red Teaming Agentic AI - Part 1

Welcome to Chapter 5 of the AI & Cybersecurity Learning Series by KK Mookhey! This chapter marks the beginning of an intensive deep dive into red teaming agentic AI systems. Following the Cloud Security Alliance's (CSA) Agentic AI Red Teaming Guide, we'll systematically test and exploit vulnerabilities in autonomous agents—then build robust guardrails to defend against these attacks. What You'll Learn: • Attack Vectors (CSA Framework): • Practical Red Teaming: • Defense Strategies: Key Vulnerabilities Demonstrated: • Authorization Hijacking • Goal Manipulation • Memory Manipulation CSA Agentic AI Red Teaming Guide: This video follows the structured approach from the Cloud Security Alliance's official red teaming framework for agentic AI systems. The guide provides comprehensive attack vectors and mitigation strategies for securing autonomous agents. Attack Vectors Covered in Part 1: ✅ Authorization and Control Hijacking ✅ Goal and Instruction Manipulation ✅ Memory and Context Manipulation Technical Architecture: • Framework: LangChain for agent orchestration • LLM: OpenAI GPT-4 for reasoning engine • UI: Streamlit for interactive testing • Memory: Scratchpad (temporary, non-persistent) • Execution Loop: Reason-Act-Observe cycle • Tools: Custom Python functions with subprocess calls Critical Insights: • Hardcoding roles is a temporary fix; production requires proper session management • Role-based permissions must be enforced at agent initialization • Prompt hardening alone is insufficient without access control • Memory manipulation exploits conversational context • Defense-in-depth requires multiple complementary guardrails • LLM-based input validation adds crucial security layer What's Next? Chapter 5.2 will cover: • Knowledge-base poisoning attacks • Exploiting hallucinations for malicious purposes • Supply chain vulnerabilities in agent systems • Advanced guardrails and detection mechanisms Code Repository: Complete vulnerable and secured agent code available in Google Doc- https://docs.google.com/document/d/1NkGZthHxy3QWdkyeDKC_TB2gFOk2H0n-BKZVZjXwq-Q/edit?tab=t.6ypecfaymbgh CSA Agentic AI Red Teaming Guide reference materials Role-based access control implementations Prompt hardening templates and examples About the Instructor: KK Mookhey leverages 25+ years of cybersecurity expertise to teach offensive and defensive techniques for securing agentic AI systems using industry-standard frameworks. Connect with KK on https://www.linkedin.com/in/kkmookhey/ Course Series Progress: This is Chapter 5.1 of our AI & Cybersecurity Learning Series. Chapter 1 to 4: https://www.youtube.com/watch?v=caSd12M5Axk&list=PLXVUBNOa2d7YyqWr_DgUHw7RwQLE7P24m&index=5 Chapter 5.1: Red Teaming Agentic AI - Part 1 ← You Are Here Chapter 5.2: Red Teaming Part 2 and Part 3(Coming Soon) Join the Conversation: • What other attack vectors would you test on this agent? • How would you improve the guardrails demonstrated? • Have you encountered similar vulnerabilities in production? • Share your experiences with agentic AI security! Timestamps: 00:00 - Introduction: Deep Dive into Red Teaming Agentic AI 00:40 - CSA Agentic AI Red Teaming Guide Overview 01:19 - Building a Tiered SOC Analyst Agent 01:57 - Attack Vector 1: Agent Authorization and Control Hijacking 02:31 - Code Walkthrough: Vulnerable Agent Setup 03:08 - Tool Definitions: Get Ticket Details vs Run Vulnerability Scan 03:47 - System Prompt Analysis: Tier 1 Restrictions 04:19 - Agent Initialization and Execution 05:08 - Initial Testing: Tool Discovery and Information Disclosure 05:52 - Attempting Privilege Escalation Attacks 07:31 - Successful Authorization Bypass: Escalation to Tier 2 09:06 - Attack Vector 2: Agent Goal and Instruction Manipulation 09:45 - Pause & Think: What Guardrails Would You Implement? 10:17 - Implementing Guardrails: Role-Based Tool Permissions 11:34 - Testing Fixed Code: Blocking Escalation Attempts 12:30 - Verification: Agent Now Respects Role Boundaries 13:11 - Prompt Hardening: Rejecting Goal Changes 13:54 - Attack Vector 3: Memory and Context Manipulation 15:05 - CSA Guide Definition: Mission and Purpose Manipulation 15:40 - Memory Manipulation: Overriding Scratchpad Instructions 16:11 - Conversation History Exploitation Techniques 16:49 - Summary: Three Critical Attack Vectors 17:22 - Defense in Depth: Multi-Layered Guardrail Strategy 17:54 - Preview: Knowledge-Base Poisoning, Hallucination, Supply Chain Attacks 18:27 - Conclusion and Next Steps Network Intelligence - Advanced training in securing autonomous AI systems using industry frameworks. #RedTeaming #AgenticAI #AIandCybersecurity #CSA #CloudSecurityAlliance #PromptInjection #PrivilegeEscalation #AuthorizationBypass #GoalManipulation #MemoryManipulation #AIAgents #LLMSecurity #Cybersecurity #KKMookhey #NetworkIntelligence #AIRedTeam #SecureAI #RoleBasedAccessControl #DefenseInDepth #OWASPTop10