Driver, executors, and cluster

A Spark application relies on a driver, which runs the main program and plans the work, and on executors, processes spread across the nodes of a cluster that run the tasks on data partitions. A cluster manager (YARN, Kubernetes, standalone mode) allocates the resources.

Related concepts

The whole glossary