Learning PySpark. Build data-intensive applications locally and deploy at scale using the combined powers of Python and Spark 2.0

Apache Spark is an open source framework for efficient cluster computing with a strong interface for data parallelism and fault tolerance. This book will show you how to leverage the power of Python and put it to use in the Spark ecosystem.

• You will get familiar with the modules available in PySpark.
• You will learn how to abstract data with RDDs and DataFrames and understand the streaming capabilities of PySpark.

You will start by getting a firm understanding of the Spark 2.0 architecture and how to set up a Python environment for Spark. Also, you will get a thorough overview of machine learning capabilities of PySpark using ML and MLlib, graph processing using GraphFrames, and polyglot persistence using Blaze.

• Finally, you will learn how to deploy your applications to the cloud using the spark-submit command.

By the end of this book, you will have established a firm understanding of the Spark Python API and how it can be used to build data-intensive applications.