Azure Databricks for Stream Processing by Dustin Vannoy

Learn about Azure Databricks for Stream Processing (real-time) using Azure Data Lake, Apache Kafka, Azure Event Hubs and basic Python with Dustin Vannoy. The Microsoft Data and AI South Florida user group host this event part of a Worldwide Presentation Series by presenters from around the world. - Title: Azure Databricks for Stream Processing by Dustin Vannoy - Topics Covered: . Azure Databricks . Azure Data Lake . Apache Kafka . Azure Event Hubs . Python - Abstract: This session exposes beginners and experienced tech professionals to techniques on how to implement real-time stream processes using different Azure services. We will talk about why we need to shift some of our workloads from batch data jobs to streaming in real-time. We'll dive into how Spark Structured Streaming in Azure Databricks enables this along with streaming data systems such as Kafka and Azure Event Hubs. We will discuss the concepts, how Azure Databricks enables stream processing, using Azure Data Lake for storage, and review Python code examples on a sample data set. In today’s world of data science and data engineering, we often default to processing in nightly or hourly batches, but that pattern is not enough anymore. Our customers and business leaders see information is being created all the time and realize it should be available much sooner. While the move to stream processing adds complexity, the tools we have available make it achievable for teams of any size. - Biography: . Dustin Vannoy: Dustin Vannoy is a data engineering consultant and trainer. His specialties are modern data pipelines, data lakes, and data warehouses. Technologies he covers include Apache Spark, Apache Kafka, Azure, AWS, Python, Scala, and SQL. He likes to learn new technology and share knowledge with others. You can find more of his content at dustinvannoy.com. - RSVP Now and Bring a Friend: http://bit.ly/3r9d6fg Connect with our supporters who make it all possible ........................................................... * Dustin Vannoy - Blog : https://dustinvannoy.com/ - LinkedIN: https://www.linkedin.com/in/dustinvannoy/ - Twitter : https://twitter.com/dustinvannoy @DustinVannoy .......................................................... * Join DataAISF - Microsoft Data and AI South Florida - YouTube - https://bit.ly/2RsbCgY​ - Subscribe Now - Twitter - https://bit.ly/3crVYKe​ - Follow Us - MeetUp - https://bit.ly/2K9deI7​ - Register - DATE & TIME: Date: Wed - Feb 17th, 2021 Time: 6:00 pm US EST (GMT -5) - LOCATION: Subscribe to the YouTube channel & Click on the Bell https://bit.ly/2RsbCgY​ RSVP and Bring a Friend. * Join DataAISF - Microsoft Data and AI South Florida - YouTube - https://bit.ly/2RsbCgY​ - Subscribe Now - Twitter - https://bit.ly/3crVYKe​ - Follow Us - MeetUp - https://bit.ly/2K9deI7​ - Register - LinkedIn - https://bit.ly/3col1Or​ - Connect with Us . .