spark-ExecutorBackend.adoc

ExecutorBackend — Pluggable Executor Backends

ExecutorBackend is a pluggable interface that TaskRunners use to send task status updates to a scheduler.

Figure 1. ExecutorBackend receives notifications from TaskRunners

Note	TaskRunner manages a single individual task and is managed by an Executor to launch a task.

Caution

FIXME What is "a scheduler" in this context?

It is effectively a bridge between the driver and an executor, i.e. there are two endpoints running.

There are three concrete executor backends:

1. CoarseGrainedExecutorBackend

2. LocalSchedulerBackend (for local run mode)

3. MesosExecutorBackend

ExecutorBackend Contract

trait ExecutorBackend {
  def statusUpdate(taskId: Long, state: TaskState, data: ByteBuffer): Unit
}

Note	ExecutorBackend is a private[spark] contract.

Table 1. ExecutorBackend Contract

Method	Description
statusUpdate	Used when TaskRunner is requested to run a task (to send task status updates).