Concurrency & Threading

Introduction

The game engine provides general-purpose multi-threading through the Job System (code). This is a recommended approach for compute-heavy tasks which may slow down the game. When a 'job' is launched, it is scheduled to be executed on an available thread at some time in the future.

The job system uses Java's CompletableFuture class to return job results. We recommend reading a tutorial or the documentation before using the job system.

Concurrency

Do not use jobs purely to run code concurrently, i.e. running two things at the same time. The update pattern (see Entity Component System) is designed to allow everything in the game to run 'concurrently' by computing one step at a time. If you want to run two things at the same time, consider just doing this in update().

Do not use jobs purely to create delays. It may be tempting when creating some delay to use Thread.sleep(delay) or equivalent on a separate thread. This is computationally expensive and unnecessary. Consider using the update() function together with class variables which store game time. For example, the following component would print to the console every 1000ms:

class PrintComponent extends Component {
  private long lastTime = 0L;

  @Override
  public void update() {
    long currentTime = ServiceLocator.getTimeSource().getTime();
    if (currentTime - lastTime >= 1000L) {
       System.out.println("Hello!");
       lastTime = currentTime;
    }
  }
}

Usage

It's important to consider whether your job is blocking or non-blocking. A blocking job is one that stops execution while waiting on something, such as a delay (Thread.sleep()) or an I/O operation (accessing a file, user input, networking).

Non-blocking jobs

Below is an example of creating a job that performs an expensive path-finding calculation for an NPC.

public class PathFindComponent extends Component {
  private Path currentPath;
  private CompletableFuture<Path> nextPathFuture;

  @Override
  public void create() {
    // Start calculating the next path
    nextPathFuture = JobSystem.launch(this::findNewPath);
  }

  @Override
  public void update() {
    if (nextPathFuture.isDone()) {
      currentPath = nextPathFuture.join();
      // Start calculating the next path
      JobSystem.launch(this::findNewPath);
    } else {
      // Continue moving along the path
      moveAlong(currentPath);
    }
  }

  Path findNewPath() {
    // This method is running on a separate thread, be careful 
    // about accessing class variables!
    return expensivePathCalculation();
  }
}

Blocking jobs

Blocking jobs have significantly more overhead than non-blocking jobs since they need a dedicated thread per job. Launching blocking jobs should not be done frequently, such as on each update() call or in many entities. However, using a blocking job is preferable to blocking in the main thread which would block the entire game. Below is an example of loading a large music file asynchronously, then playing it once it's loaded.

public class MusicComponent extends Component {
  private CompletableFuture<Music> musicFuture;
  private Music music;

  @Override
  public void create() {
    // Start reading the file in a blocking job
    musicFuture = JobSystem.launchBlocking(() -> this.readMusic("beats.wav"));
  }

  @Override
  public void update() {
    if (music == null && musicFuture.isDone()) {
      // Finished loading music from the job
      music = musicFuture.join();
    } else {
      music.play();
    }
  }

  Music readMusic(String filename) {
    // Start loading a large music file
    ResourceService resourceService = ServiceLocator.getResourceService();
    resourceService.loadMusic(new String[] {filename});
    // Block until fully loaded
    resourceService.loadAll();
    return resourceService.getAsset(filename, Music.class);
  }
}

Behind the Scenes

Why isn't the job system used in the base game?

Despite the enormous performance benefits, the base game and underlying engine don't make use of the job system or any other multi-threading. Doing so would require anyone making changes to the game to write only thread-safe code. An understanding of concurrency/threading is not a prerequisite to working with this engine, and concurrency bugs are very difficult to track down and fix. If you are interested in learning how concurrency is integrated into AAA game engines, check out the recommended resources!

What's the benefit of using a job system over just creating threads?

Creating and destroying threads is a computationally expensive operation. When this is done every frame of the game (roughly 16ms) for potentially many entities, that overhead can significantly affect performance. This is why in pratice, game engines and most other multi-threaded programs use thread pools instead. On game launch, we create a pool of threads that don't get destroyed until the game ends. When a new thread is requested, we just re-use a thread from the pool, circumventing the creation/deletion cost.

There may also be many more jobs than physical CPU cores, but there is no performance benefit to having more threads than CPU cores (and context switching between threads also takes time). This is why many languages support virtual threads or some other form of user-space concurrency like coroutines. Unfortunately Java does not have native support for this, but does provide similar functionality in ForkJoinPool, which spawns one thread per CPU core. Tasks are added to a queue, threads take tasks from that queue and run them. The game engine's job system uses this internally for non-blocking jobs.

Further Resources

• Introduction to Concurrency: Chapter 4, Game Engine Architecture (3rd Edition)
• Parallelizing the Naughty Dog Engine (Video)
• Parallel Game Engine Design (Video)
• Concurrency in Game Engines: Chapter 8.6, Game Engine Architecture (3rd Edition)