nvme-multipath: fix possible I/O hang when paths are updated [Linux 4.19.72]

This Linux kernel change "nvme-multipath: fix possible I/O hang when paths are updated" is included in the Linux 4.19.72 release. This change is authored by Anton Eidelman <anton [at] lightbitslabs.com> on Mon Aug 12 23:00:36 2019 +0300. The commit for this change in Linux stable tree is 5e416b1 (patch) which is from upstream commit 504db08. The same Linux upstream change may have been applied to various maintained Linux releases and you can find all Linux releases containing changes from upstream 504db08.

nvme-multipath: fix possible I/O hang when paths are updated

[ Upstream commit 504db087aaccdb32af61539916409f7dca31ceb5 ]

nvme_state_set_live() making a path available triggers requeue_work
in order to resubmit requests that ended up on requeue_list when no
paths were available.

This requeue_work may race with concurrent nvme_ns_head_make_request()
that do not observe the live path yet.
Such concurrent requests may by made by either:
- New IO submission.
- Requeue_work triggered by nvme_failover_req() or another ana_work.

A race may cause requeue_work capture the state of requeue_list before
more requests get onto the list. These requests will stay on the list
forever unless requeue_work is triggered again.

In order to prevent such race, nvme_state_set_live() should
synchronize_srcu(&head->srcu) before triggering the requeue_work and
prevent nvme_ns_head_make_request referencing an old snapshot of the
path list.

Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>

There is one line of Linux source code added/deleted in this change. Code changes to Linux kernel are as follows.

 drivers/nvme/host/multipath.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 05d6371..f57feb8 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -323,6 +323,7 @@ static void nvme_mpath_set_live(struct nvme_ns *ns)
                 "failed to create id group.\n");
    }

+   synchronize_srcu(&ns->head->srcu);
    kblockd_schedule_work(&ns->head->requeue_work);
 }

Leave a Reply

Your email address will not be published. Required fields are marked *