diff options
| author | Paul E. McKenney <paulmck@kernel.org> | 2021-12-14 13:15:18 -0800 |
|---|---|---|
| committer | Paul E. McKenney <paulmck@kernel.org> | 2022-02-08 10:12:28 -0800 |
| commit | a16578dd5e3a44b53ca0699ac2971679dab97484 (patch) | |
| tree | b1ccb9fd4b7a38ffcb6c93114059018b1ee7165d /kernel/rcu/tree.h | |
| parent | 0cabb47af3cfaeb6007ba3868379bbd4daee64cc (diff) | |
rcu: Rework rcu_barrier() and callback-migration logic
This commit reworks rcu_barrier() and callback-migration logic to
permit allowing rcu_barrier() to run concurrently with CPU-hotplug
operations. The key trick is for callback migration to check to see if
an rcu_barrier() is in flight, and, if so, enqueue the ->barrier_head
callback on its behalf.
This commit adds synchronization with RCU's CPU-hotplug notifiers. Taken
together, this will permit a later commit to remove the cpus_read_lock()
and cpus_read_unlock() calls from rcu_barrier().
[ paulmck: Updated per kbuild test robot feedback. ]
[ paulmck: Updated per reviews session with Neeraj, Frederic, Uladzislau, and Boqun. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Diffstat (limited to 'kernel/rcu/tree.h')
| -rw-r--r-- | kernel/rcu/tree.h | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 4b4bcef8a974..84362951ed9e 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -188,6 +188,8 @@ struct rcu_data { bool rcu_forced_tick_exp; /* ... provide QS to expedited GP. */ /* 4) rcu_barrier(), OOM callbacks, and expediting. */ + raw_spinlock_t barrier_lock; /* Protects ->barrier_seq_snap. */ + unsigned long barrier_seq_snap; /* Snap of rcu_state.barrier_sequence. */ struct rcu_head barrier_head; int exp_dynticks_snap; /* Double-check need for IPI. */ |