spark-BroadcastFactory.adoc

BroadcastFactory — Pluggable Broadcast Variable Factories

BroadcastFactory is the contract for factories of broadcast variables in Apache Spark.

Note	As of Spark 2.0, it is no longer possible to plug a custom BroadcastFactory in, and TorrentBroadcastFactory is the one and only known implementation.

BroadcastFactory is exclusively used and instantiated inside of BroadcastManager.

package org.apache.spark.broadcast

trait BroadcastFactory {
  def initialize(isDriver: Boolean, conf: SparkConf, securityMgr: SecurityManager): Unit
  def newBroadcast[T: ClassTag](value: T, isLocal: Boolean, id: Long): Broadcast[T]
  def unbroadcast(id: Long, removeFromDriver: Boolean, blocking: Boolean): Unit
  def stop(): Unit
}

Note	BroadcastFactory is a private[spark] contract.

Table 1. BroadcastFactory Contract

Method	Description
initialize	Used when…​FIXME
newBroadcast	Used when…​FIXME
unbroadcast	Used when…​FIXME
stop	Used when…​FIXME

Note	TorrentBroadcastFactory is the one and only known implementation of BroadcastFactory contract in Apache Spark.