The Concurrency API
7.1 Task-Based Programming
1) Thread-Based Programming
Thread-Based Programming: A concurrency approach that directly creates and manages std::thread objects.
std::thread t(doAsyncWork);
The programmer is responsible for managing the underlying threads.
2) Task-Based Programming
Task-Based Programming: A concurrency approach that describes work as tasks and lets the runtime manage thread execution.
auto fut = std::async(doAsyncWork);
It provides a higher-level abstraction than direct thread management.
3) Task
Task: A callable object passed to std::async for execution.
The task represents the work to perform rather than the thread that performs it.
4) Future
Future: An object representing the eventual result of an asynchronous operation.
auto fut = std::async(doAsyncWork);
auto result = fut.get();
A future provides access to both return values and exceptions produced by the task.
5) Hardware Thread
Hardware Thread: A hardware execution resource that actually performs computation.
A CPU core may provide one or more hardware threads.
6) Software Thread
Software Thread: An operating-system-managed thread scheduled onto hardware threads.
Software threads are also called OS threads or system threads.
7) std::thread
std::thread: A C++ object that acts as a handle to an underlying software thread.
8) Oversubscription
Oversubscription: A state where the number of runnable software threads exceeds the available hardware threads.
It increases scheduling and context-switching overhead.
9) Context Switch
Context Switch: The process of suspending one software thread and running another.
Frequent context switches can reduce performance and disrupt CPU cache locality.
10) Task-Based vs Thread-Based
Task-based programming with std::async can delegate:
- Thread creation and destruction
- Thread exhaustion handling
- Oversubscription management
- Load balancing
Direct std::thread programming is appropriate when low-level thread control such as native handles, priorities, or affinities is required.
7.2 std::async Launch Policies
1) Launch Policy
Launch Policy: A policy that determines how a task passed to std::async is executed.
C++ provides std::launch::async and std::launch::deferred.
2) std::launch::async
std::launch::async: Requires the task to execute asynchronously on another thread.
auto fut = std::async(
std::launch::async,
f
);
Use it when asynchronous execution is essential.
3) std::launch::deferred
std::launch::deferred: Delays execution until get() or wait() is called on the associated future.
The task executes synchronously in the thread making that call.
If neither operation occurs, the task may never execute.
4) Default Launch Policy
Default Launch Policy: Allows std::async to choose between asynchronous and deferred execution.
Conceptually:
std::launch::async | std::launch::deferred
Therefore, a default std::async call does not guarantee concurrent execution.
5) std::future_status
std::future_status: Represents the state returned by timed future waiting operations.
Important states include:
std::future_status::readystd::future_status::timeoutstd::future_status::deferred
6) Detecting Deferred Execution
A zero-duration wait_for can determine whether a task is deferred.
if (fut.wait_for(0s) ==
std::future_status::deferred)
{
// task is deferred
}
Timeout-based waiting logic must account for deferred tasks.
7.3 std::thread Joinability
1) Joinable Thread
Joinable Thread: A std::thread that is associated with an underlying thread of execution.
A thread remains joinable even after its function finishes until it is joined or detached.
2) Unjoinable Thread
Unjoinable Thread: A std::thread that is not associated with a thread that still requires ownership management.
A thread is unjoinable when it is:
- Default constructed
- Moved from
- Joined
- Detached
3) join()
join(): Waits for the underlying thread to finish and makes the std::thread unjoinable.
t.join();
4) detach()
detach(): Separates the std::thread object from its underlying thread.
t.detach();
The underlying thread continues independently.
5) Joinable Thread Destruction
Destroying a joinable std::thread causes program termination.
Therefore, every execution path leaving a scope must make its std::thread objects unjoinable.
6) RAII
RAII (Resource Acquisition Is Initialization): A technique that ties resource cleanup to object lifetime.
A thread-owning RAII object can call join() or detach() automatically in its destructor.
7) Join-on-Destruction
Join-on-Destruction: Automatically calling join() when a thread-owning RAII object is destroyed.
It prevents unsafe detached access but may introduce blocking or performance problems.
8) Detach-on-Destruction
Detach-on-Destruction: Automatically calling detach() when a thread-owning RAII object is destroyed.
It can cause undefined behavior when the detached thread continues using objects whose lifetimes have ended.
7.4 Future Destructor Behavior
1) Shared State
Shared State: Storage shared between an asynchronous producer and the future receiving its result.
It contains the result or exception produced by the asynchronous operation.
Conceptually:
Producer -> Shared State -> Future
2) std::promise
std::promise: The writing end of a future-based communication channel.
A promise stores a value or exception in the shared state.
3) std::future
std::future: A single-owner handle used to access a shared state's result.
4) std::shared_future
std::shared_future: A copyable future that allows multiple objects to refer to the same shared state.
5) Normal Future Destruction
Normally, destroying a future simply destroys the future object and releases its reference to the shared state.
6) Blocking Future Destruction
The final future referring to a non-deferred task launched through std::async with std::launch::async blocks until that task completes.
This behaves similarly to an implicit join.
7) std::packaged_task
std::packaged_task: A wrapper that allows a callable object's result to be stored in a shared state.
std::packaged_task<int()> task(calcValue);
auto fut = task.get_future();
It provides another way to create a future without directly using std::async.
7.5 One-Shot Event Communication
1) Condition Variable
Condition Variable: A synchronization primitive that allows a thread to block until another thread signals that a condition may have changed.
std::condition_variable cv;
It is normally used together with a mutex.
2) notify_one()
notify_one(): Wakes one thread waiting on a condition variable.
cv.notify_one();
3) notify_all()
notify_all(): Wakes all threads waiting on a condition variable.
4) Spurious Wakeup
Spurious Wakeup: A condition-variable wait returning even though no corresponding notification occurred.
The waiting thread must verify that the desired condition is actually true.
cv.wait(lock, [] {
return condition;
});
5) Polling
Polling: Repeatedly checking a value until a condition becomes true.
while (!flag);
Polling avoids condition-variable wakeup issues but continuously consumes execution resources.
6) Void Future
Void Future: A std::future<void> used to communicate that an event occurred without transmitting a value.
std::promise<void> p;
auto fut = p.get_future();
p.set_value();
fut.wait();
The promise signals the event, and the future waits for it.
7) One-Shot Communication
One-Shot Communication: Communication that can signal an event only once.
A std::promise can be satisfied only once, making promise/future communication suitable for one-time events.
8) Multiple Waiting Tasks
A std::shared_future<void> allows multiple threads to wait for the same one-shot event.
auto sf = p.get_future().share();
Each waiting task can hold its own copy of the shared future.
7.6 std::atomic and volatile
1) std::atomic
std::atomic: A type that provides operations that other threads observe atomically.
std::atomic<int> value{0};
++value;
It is intended for data shared between concurrent threads.
2) Atomic Operation
Atomic Operation: An operation that appears indivisible to other threads.
Other threads cannot observe a partially completed atomic operation.
3) Read-Modify-Write Operation
Read-Modify-Write (RMW): An operation that reads a value, modifies it, and writes the result as one atomic operation.
Examples include atomic increment and decrement.
++value;
--value;
4) Data Race
Data Race: Concurrent access to the same memory where at least one access modifies it and the accesses are not properly synchronized.
A data race results in undefined behavior.
5) load()
load(): Atomically reads the value of a std::atomic object.
auto value = atomicValue.load();
6) store()
store(): Atomically writes a value to a std::atomic object.
atomicValue.store(10);
7) Sequential Consistency
Sequential Consistency: The default atomic memory-ordering model in which atomic operations behave as if they occur in a single globally consistent order.
It provides stronger and easier-to-reason-about ordering guarantees than relaxed memory models.
8) volatile
volatile: A qualifier indicating that accesses to an object represent observable accesses to special memory and should not be optimized away as ordinary redundant memory operations.
volatile int value;
It is not a replacement for std::atomic in multithreaded synchronization.
9) Special Memory
Special Memory: Memory whose reads or writes may have effects beyond ordinary RAM access.
A common example is memory-mapped I/O connected to hardware devices.
10) Memory-Mapped I/O
Memory-Mapped I/O: A technique where hardware registers or devices are accessed through memory addresses.
Repeated reads and writes may each have distinct effects, so they must not be eliminated as redundant.
11) std::atomic vs volatile
std::atomic: Used for synchronization and atomic access between threads.
volatile: Used when accesses to special memory must actually occur and must not be optimized away.
They solve different problems and are not interchangeable.