Spark SQL Kickstart: Your first Spark SQL application

Get hands on with Spark SQL to build your first data pipeline. In this video I walk you through how to read, transform, and write the NYC Taxi dataset which can be found on Databricks, Azure Synapse, or downloaded from the web to wherever you run Apache Spark. Once you have watched and followed along with this tutorial, go find a free dataset and try to write your own Spark application. * All thoughts and opinions are my own * For links to the code and more information on this course, you can visit my website: https://dustinvannoy.com/2023/05/18/first-spark-sql-application/ More from Dustin: Website: https://dustinvannoy.com LinkedIn: https://www.linkedin.com/in/dustinvannoy Github: https://github.com/datakickstart CHAPTERS 00:00 Intro 1:11 Spark SQL data pipeline (notebook) 11:56 Spark SQL locally 18:30 Outro