Azure Data Explorer and Databricks - Big Data Analytics

What's this session about? ========================== The video session will cover an architecture designed to ingest and analyze security logs data using Azure Databricks as an ETL tool and Azure Data Explorer as a big data analytics store; both products are well-known for efficiently processing and analyzing massive amounts of data. The session also includes features for production environments like Status monitoring, exactly once ingestion checks, Custom error-retry, and Performance metrics alert. Case Study: ============ We will assume the role of a Data Platform Architect of a global data analytics service company. As the company's business is booming, you want to implement a new system that can collect telemetry data from hundreds of thousands of devices for the growing customers. These devices generate telemetry data 24X7 across multiple continents. Members of your team, including Data analysts and Machine Learning Specialists, are tasked with analyzing data coming in as soon as possible. Speaker's Bio: ============== "Shivam Sharma", Co-founder TechScalable Shivam is an author, cloud architect, speaker, and Co-Founder at TechScalable. Being passionate about ever-evolving technology he works on Azure, GCP, Machine Learning, Data Engineering, and DevOps. He architects’ solutions on Cloud as well on-premises using a wide array of platforms/technologies. https://www.linkedin.com/in/shivam-sharma-ts/