In this video, you learn how to use PySpark dataframes methods on Databricks to perform data analysis and engineering at scale. This is the core of using Python on Spark and you need to learn the power but also the nuances involved.
Video demo notebook at:
https://github.com/bcafferky/shared/blob/master/MasterDatabricksAndSpark/Lesson_23_AW_PySpark_Dataframe_Methods_2.zip
Apache Spark Zeppelin Notebook link will be posted later.
For information on how to upload files to Databricks see:
https://www.youtube.com/watch?v=wqCCWAa6mFA
