Courses
Apache Spark
Python Vs. Scala

Python Vs. Scala

Lesson objectives

In this lesson, we will explain the following topics:

  • Compare the differences between Python and Scala in the context of Spark.
  • Understand the performance implications of using Python vs. Scala.
  • Learn about the advantages and disadvantages of each language for Spark development.

Python vs Scala

  • Python is widely used with numerous tools and libraries available.
  • Python is easier to learn than Scala; however, Scala might be more intuitive for those who prefer functional programming.
  • Finding Python developers is generally easier for companies than finding Scala developers.
  • Initially, Scala offered better performance in Apache Spark, but over time this advantage reduced, and now there’s no big difference in speed.
  • PySpark and Scala share the same Spark concepts, allowing for interchangeable use of examples from both languages without affecting learning.

Attention!

To Be a Spark Expert You Have to Be Able to Read a Little Scala Anyway! Referenced from High Performance Spark, 2nd Edition, Ch.01

Spark’s Codebase and Documentation

  • The quality of Spark’s documentation is inconsistent. Referenced from High Performance Spark, 2nd Edition, Ch.01
  • Spark’s codebase is very readable.
  • Understanding the Spark codebase benefits advanced users.

Understanding Spark Through Scala

  • Scala helps you understand Spark deeply. Referenced from High Performance Spark, 2nd Edition, Ch.01
  • Spark is written in Scala.
  • To work with Spark’s source code effectively, it’s essential to understand (read) Scala.

RDD and Scala’s Influence

  • Scala’s influence is evident in Spark’s Resilient Distributed Datasets (RDD). Referenced from High Performance Spark, 2nd Edition, Ch.01
  • RDD methods are similar to Scala’s collection tools.
  • Functions like map, filter, and reduce are similar in both.
  • Knowing Scala makes it easier to understand how RDDs work.

Spark as a Functional Framework

  • Spark uses functional programming principles.
  • Concepts like immutability and lambda are key.
  • Understanding functional programming helps in using Spark well. Referenced from High Performance Spark, 2nd Edition, Ch.01

Watch on Youtube

Watch on our Servers

You can download the videog the link and chose save link as: Download Video