perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data [Linux 5.2]

perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data [Linux 5.2]

This Linux kernel change "perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data" is included in the Linux 5.2 release. This change is authored by Peter Zijlstra <peterz [at] infradead.org> on Fri May 17 13:52:33 2019 +0200. The commit for this change in Linux stable tree is 4d839dd (patch).

perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data

We must use {READ,WRITE}_ONCE() on rb->user_page data such that
concurrent usage will see whole values. A few key sites were missing
this.

Suggested-by: Yabin Cui <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: Alexander Shishkin <[email protected]>
Cc: Arnaldo Carvalho de Melo <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Vince Weaver <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Fixes: 7b732a750477 ("perf_counter: new output ABI - part 1")
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>

There are 6 lines of Linux source code added/deleted in this change. Code changes to Linux kernel are as follows.

 kernel/events/ring_buffer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 4b5f8d93..7a0c73e 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -100,7 +100,7 @@ static void perf_output_put_handle(struct perf_output_handle *handle)
     * See perf_output_begin().
     */
    smp_wmb(); /* B, matches C */
-   rb->user_page->data_head = head;
+   WRITE_ONCE(rb->user_page->data_head, head);

    /*
     * We must publish the head before decrementing the nest count,
@@ -496,7 +496,7 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
        perf_event_aux_event(handle->event, aux_head, size,
                     handle->aux_flags);

-   rb->user_page->aux_head = rb->aux_head;
+   WRITE_ONCE(rb->user_page->aux_head, rb->aux_head);
    if (rb_need_aux_wakeup(rb))
        wakeup = true;

@@ -528,7 +528,7 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size)

    rb->aux_head += size;

-   rb->user_page->aux_head = rb->aux_head;
+   WRITE_ONCE(rb->user_page->aux_head, rb->aux_head);
    if (rb_need_aux_wakeup(rb)) {
        perf_output_wakeup(handle);
        handle->wakeup = rb->aux_wakeup + rb->aux_watermark;

Leave a Reply

Your email address will not be published. Required fields are marked *