Xumin Xu

man sitting at computer desk in home office
Databricks Spark jobs optimization techniques: Pandas UDF
In this tutorial, you'll learn about leveraging Pandas UDF as a technique for Databricks Spark jobs optimization.
Databricks Spark jobs optimization techniques: Multi-threading
The Multi-threading technique can help optimize Databricks Spark jobs, saving time and creating a better load balance.
code on laptop in office
Databricks Spark jobs optimization techniques: Shuffle partition technique (Part 1)
Here we cover the key ideas behind shuffle partition, how to set the right number of partitions, and how to use these to optimize Spark jobs.