perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data [Linux 4.14.129]

perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data [Linux 4.14.129]

This Linux kernel change "perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data" is included in the Linux 4.14.129 release. This change is authored by Peter Zijlstra <peterz [at] infradead.org> on Fri May 17 13:52:33 2019 +0200. The commit for this change in Linux stable tree is a9a8310 (patch) which is from upstream commit 4d839dd. The same Linux upstream change may have been applied to various maintained Linux releases and you can find all Linux releases containing changes from upstream 4d839dd.

perf/ring-buffer: Always use {READ,WRITE}_ONCE() for rb->user_page data

[ Upstream commit 4d839dd9e4356bbacf3eb0ab13a549b83b008c21 ]

We must use {READ,WRITE}_ONCE() on rb->user_page data such that
concurrent usage will see whole values. A few key sites were missing
this.

Suggested-by: Yabin Cui <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: Alexander Shishkin <[email protected]>
Cc: Arnaldo Carvalho de Melo <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Vince Weaver <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Fixes: 7b732a750477 ("perf_counter: new output ABI - part 1")
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>

There are 6 lines of Linux source code added/deleted in this change. Code changes to Linux kernel are as follows.

 kernel/events/ring_buffer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index aef2af8..f3a69a4 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -101,7 +101,7 @@ static void perf_output_put_handle(struct perf_output_handle *handle)
     * See perf_output_begin().
     */
    smp_wmb(); /* B, matches C */
-   rb->user_page->data_head = head;
+   WRITE_ONCE(rb->user_page->data_head, head);

    /*
     * We must publish the head before decrementing the nest count,
@@ -489,7 +489,7 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
                             handle->aux_flags);
    }

-   rb->user_page->aux_head = rb->aux_head;
+   WRITE_ONCE(rb->user_page->aux_head, rb->aux_head);
    if (rb_need_aux_wakeup(rb))
        wakeup = true;

@@ -520,7 +520,7 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size)

    rb->aux_head += size;

-   rb->user_page->aux_head = rb->aux_head;
+   WRITE_ONCE(rb->user_page->aux_head, rb->aux_head);
    if (rb_need_aux_wakeup(rb)) {
        perf_output_wakeup(handle);
        handle->wakeup = rb->aux_wakeup + rb->aux_watermark;

Leave a Reply

Your email address will not be published. Required fields are marked *