This video shows you how to read data from a SQL Database from Databricks/Spark using Python using parallel reads so you get back partitioned data. SQL databases are the most popular platform for storing data so you are going to need this in your toolbelt. This video picks up where Lesson 30 left off and provides better performance.
Join my Patreon Community and Watch this Video without Ads!
https://www.patreon.com/bePatron?u=63260756
Twitter: @BryanCafferky
Notebook with Code at:
https://github.com/bcafferky/shared/blob/master/MasterDatabricksAndSpark/Lesson_31_Read_From_SQL_Database_in_parallel.zip
Video on Creating an Azure SQL Database
https://www.youtube.com/watch?v=VnU5-erCIC0
Video on Creating an Azure Key Vault
https://www.youtube.com/watch?v=Lwhu5GD5NDY&t=2s
Download
0 formats
No download links available.
Master Databricks and Apache Spark Step by Step: Lesson 31 - PySpark: Parallel Database Queries | NatokHD