io_uring: fix hanging iopoll in case of -EAGAIN

[ Upstream commit cd664b0e35cb1202f40c259a1a5ea791d18c879d ]

io_do_iopoll() won't do anything with a request unless
req->iopoll_completed is set. So io_complete_rw_iopoll() has to set
it, otherwise io_do_iopoll() will poll a file again and again even
though the request of interest was completed long time ago.

Also, remove -EAGAIN check from io_issue_sqe() as it races with
the changed lines. The request will take the long way and be
resubmitted from io_iopoll*().

io_kiocb's result and iopoll_completed")

Fixes: bbde017a32b3 ("io_uring: add memory barrier to synchronize
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 1829be7f63a35f44d7690ffc0c60b4790d4642eb..4ab1728de247c712486744d60179e98a9977f47e 100644 (file)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1942,10 +1942,8 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res, long res2)
 
        WRITE_ONCE(req->result, res);
        /* order with io_poll_complete() checking ->result */
-       if (res != -EAGAIN) {
-               smp_wmb();
-               WRITE_ONCE(req->iopoll_completed, 1);
-       }
+       smp_wmb();
+       WRITE_ONCE(req->iopoll_completed, 1);
 }
 
 /*
@@ -5425,9 +5423,6 @@ static int io_issue_sqe(struct io_kiocb *req, const struct io_uring_sqe *sqe,
        if ((ctx->flags & IORING_SETUP_IOPOLL) && req->file) {
                const bool in_async = io_wq_current_is_worker();
 
-               if (req->result == -EAGAIN)
-                       return -EAGAIN;
-
                /* workqueue context doesn't hold uring_lock, grab it now */
                if (in_async)
                        mutex_lock(&ctx->uring_lock);