Creating a Serverless Apache Kafka(MSK) publisher using AWS Lambda

Kafka is a top-notch industry platform for streaming data processing at scale. No surprise that first-class citizens of Kafka world are 24/7-running producer/consumer applications (e.g. classical servers, k8s-pods, etc.). But what about the rapidly rising world of AWS Serverless ecosystem? In this video ,let’s explore how to publish messages in MSK Kafka Topic from Serverless computing platform, AWS Lambda. Prerequisites: ---------------------- Learn AWS VPC , Subnet , Route Table, Internet Gateway with Hands-on Demo https://youtu.be/CYi92erGEcw VPC with Public and Private Subnets in-depth intuition with Hands On https://youtu.be/OjeTLHVFiKg AWS VPC NAT Gateway In-depth intuition https://youtu.be/2OrZd4p6EfI AWS VPC ENDPOINT in depth intuition & hands on https://youtu.be/FTzqLRkyiT0 Using Amazon VPC Endpoints to Access DynamoDB in-depth https://youtu.be/-nX2k7AfDyc Getting Started with AWS Managed Streaming for Kafka with in-depth service setup https://youtube.com/watch?v=BFKmQAafE_c&feature=shares Steps for the Lab: ------------------------------- https://github.com/SatadruMukherjee/Data-Preprocessing-Models/blob/main/Publish%20Message%20in%20MSK%20Cluster%20from%20AWS%20Lambda.txt You can download the Lambda Layer zip from here : ------------------------------------------------------------------------------------------------ https://github.com/SatadruMukherjee/Data-Preprocessing-Models/blob/main/kafka_yt_demo.zip Check this playlist for more Data Engineering related videos: https://youtube.com/playlist?list=PLjfRmoYoxpNopPjdACgS5XTfdjyBcuGku Apache Kafka form scratch https://youtube.com/playlist?list=PLjfRmoYoxpNrs0VmIq6mOTqXP52RfZdRf Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation-- https://doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510 🙏🙏🙏🙏🙏🙏🙏🙏 YOU JUST NEED TO DO 3 THINGS to support my channel LIKE SHARE & SUBSCRIBE TO MY YOUTUBE CHANNEL