In this video, we will discuss about the commonly asked Spark interview Question. We also often face this scenario in production. In this video, we will learn about Spark Optimization technique. We do performance testing of spark code using inferschema and predefined structtype schema and understand the best approach to follow. We use Databricks Community edition for this demo.
Linkedin profile:
https://www.linkedin.com/in/azarudeen-s-83652474/
FB page:
https://www.facebook.com/LearntoSpark-104523781121008/
Sample Dataset and Code Snippet:
https://www.learntospark.com/2020/10/spark-optimization-technique-inferschema.html
Blog link to learn more on Spark:
www.learntospark.com
