3_10_threads

Task 3.10: Multithreading and parallelism

One of main Rust's design goals is a concurrency. Rust has a strong opinion about that, while allows different concurrent models to coexist.

Threads

Rust has built-in support for native threads in form of the std::thread module of its standard library.

Traditionally, threads are used for solving CPU-bound problems, as they allow to execute tasks in parallel. However, in practice, threads are often used to solve I/O-bound problems too, especially when asynchronous I/O is not supported well (which is true for Rust std library at the moment).

crossbeam crate also provides implementation of scoped threads, which allow to borrow values from a stack. They are also available in form of std::thread::scope, as of Rust 1.63.

For better understanding Rust threads design, concepts, usage, and features (especially TLS is important and widely used one), read through the following articles:

• Rust Book: 16.1. Using Threads to Run Code Simultaneously
• Rust By Example: 20.1. Threads
• Official std::thread docs
• Nicky Meuleman: Multithreading in Rust

Synchronization

The threads synchronization is a wide topic, but generally it's done via atomic operations, shared state with an exclusive access, or by threads communication. Rust has built-in support for all of them.

Atomic operations are represented by std::sync::atomic module of Rust standard library (and, additionally, atomic crate).

Exclusive access may be controlled via primitives of std::sync module of Rust standard library.

Threads communication is commonly represented via channels and is implemented in std::sync::mpsc module of Rust standard library.

Despite that, there is also the crossbeam crate, providing more feature-rich and optimized concurrency and synchronization primitives. The most notable is crossbeam-channel as an enhancement of std channel implementations.

For better understanding and familiarity with Rust synchronization primitives design, concepts, usage, and features, read through the following articles:

• Rust Book: 16.2. Using Message Passing to Transfer Data Between Threads
• Rust Book: 16.3. Shared-State Concurrency
• Rust Blog: Fearless Concurrency with Rust
• Official std::sync docs
• Official std::sync::atomic docs
• Official std::sync::mpsc docs
• Official atomic crate docs
• Official crossbeam-channel crate docs
• Nicky Meuleman: Multithreading in Rust
• Carl Fredrik Samson: Explaining Atomics in Rust
• Aleksey Kladov: Mutexes Are Faster Than Spinlocks
• Mara Bos: Comparing Rust's and C++'s Concurrency Library
• Mahmoud Al-Qudsi: Implementing truly safe semaphores in rust
• Michael Snoyman: My Best and Worst Deadlock in Rust

Parallelism

The important concept to understand is how concurrency and parallelism differ.

Rust ecosystem has support for parallelism in form of rayon and dpc-pariter crates, which make it easy to convert a sequential iterator to execute in parallel threads.

Another way to perform parallel data processing without using threads is SIMD instructions usage. If an algorithm is parallelizable enough, applying SIMD instructions may increase performance drastically. Rust ecosystem provides basic support for SIMD instructions in a form of packed_simd crate.

For better understanding and familiarity with parallelism in Rust, read through the following articles:

• Nicky Meuleman: Concurrent vs parallel
• Official rayon crate docs
• rayon crate FAQ
• rayon crate demos
• Kofi Otuo: Implementing data parallelism with Rayon Rust
• Dawid Ciężarkiewicz: Adding parallelism to your Rust iterators with dpc-pariter
• Official dpc-pariter crate docs
• Rust Edition Guide: 3.9. SIMD for faster computing
• Official packed_simd crate docs
• vgatherps: Parsing numbers into base-10 decimals with SIMD

Multiprocessing

Multiprocessing is a system that has more than one or two processors. In Multiprocessing, CPUs are added for increasing computing speed of the system. Because of Multiprocessing, There are many processes are executed simultaneously.

Multiprocessing vs Multithreading

Parameter	Multiprocessing	Multithreading
Basic	Multiprocessing helps you to increase computing power.	Multithreading helps you to create computing threads of a single process to increase computing power.
Execution	It allows you to execute multiple processes concurrently.	Multiple threads of a single process are executed concurrently.
CPU switching	In Multiprocessing, CPU has to switch between multiple programs so that it looks like that multiple programs are running simultaneously.	In multithreading, CPU has to switch between multiple threads to make it appear that all threads are running simultaneously.
Creation	The creation of a process is slow and resource-specific.	The creation of a thread is economical in time and resource.
Classification	Multiprocessing can be symmetric or asymmetric.	Multithreading is not classified.
Memory	Multiprocessing allocates separate memory and resources for each process or program.	Multithreading threads belonging to the same process share the same memory and resources as that of the process.
Pickling objects	Multiprocessing relies on pickling objects in memory to send to other processes.	Multithreading avoids pickling.
Program	Multiprocessing system allows executing multiple programs and tasks.	Multithreading system executes multiple threads of the same or different processes.

Task

Estimated time: 1 day

Write a program with the following workflow:

• Producer is a separate thread, which continuously generates square matrixes of random u8 elements and size 4096.
• Consumer is a separate thread, which takes a generated matrix, counts sum of all its elements and prints the sum to STDOUT.
• There are only 1 Producer and 2 Consumers.
• Counting sum of matrix elements should be parallelized.

Questions

After completing everything above, you should be able to answer (and understand why) the following questions:

• What is concurrency? What is parallelism? How do they relate to each other and how do they differ?
• How parallelism is represented in Rust? Which are common crates for using it?
• What are the main ways of threads synchronization in Rust? Which advantages and disadvantages does each one have? What are the use-cases for each one?