kernel/linux.git/crypto/crypto_engine.c, branch v6.1.168

crypto: engine - fix crypto_queue backlog handling

2023-05-17T09:53:40+00:00

[ Upstream commit 4140aafcff167b5b9e8dae6a1709a6de7cac6f74 ] CRYPTO_TFM_REQ_MAY_BACKLOG tells the crypto driver that it should internally backlog requests until the crypto hw's queue becomes full. At that point, crypto_engine backlogs the request and returns -EBUSY. Calling driver such as dm-crypt then waits until the complete() function is called with a status of -EINPROGRESS before sending a new request. The problem lies in the call to complete() with a value of -EINPROGRESS that is made when a backlog item is present on the queue. The call is done before the successful execution of the crypto request. In the case that do_one_request() returns < 0 and the retry support is available, the request is put back in the queue. This leads upper drivers to send a new request even if the queue is still full. The problem can be reproduced by doing a large dd into a crypto dm-crypt device. This is pretty easy to see when using Freescale CAAM crypto driver and SWIOTLB dma. Since the actual amount of requests that can be hold in the queue is unlimited we get IOs error and dma allocation. The fix is to call complete with a value of -EINPROGRESS only if the request is not enqueued back in crypto_queue. This is done by calling complete() later in the code. In order to delay the decision, crypto_queue is modified to correctly set the backlog pointer when a request is enqueued back. Fixes: 6a89f492f8e5 ("crypto: engine - support for parallel requests based on retry mechanism") Co-developed-by: Sylvain Ouellet Signed-off-by: Sylvain Ouellet Signed-off-by: Olivier Bacon Signed-off-by: Herbert Xu Signed-off-by: Sasha Levin

crypto: engine - Use crypto_request_complete

2023-05-17T09:53:40+00:00

[ Upstream commit 6909823d47c17cba84e9244d04050b5db8d53789 ] Use the crypto_request_complete helper instead of calling the completion function directly. Signed-off-by: Herbert Xu Stable-dep-of: 4140aafcff16 ("crypto: engine - fix crypto_queue backlog handling") Signed-off-by: Sasha Levin

crypto: engine - Add parameter description in crypto_transfer_request() kernel-doc comment

2022-04-08T08:13:30+00:00

Add the description of @need_pump in crypto_transfer_request() kernel-doc comment to remove warning found by running scripts/kernel-doc, which is caused by using 'make W=1'. crypto/crypto_engine.c:260: warning: Function parameter or member 'need_pump' not described in 'crypto_transfer_request' Reported-by: Abaci Robot Signed-off-by: Yang Li Signed-off-by: Herbert Xu

crypto: engine - check if BH is disabled during completion

2022-03-02T22:47:49+00:00

When doing iperf over ipsec with crypto hardware sun8i-ce, I hit some spinlock recursion bug. This is due to completion function called with enabled BH. Add check a to detect this. Fixes: 735d37b5424b ("crypto: engine - Introduce the block request crypto engine framework") Signed-off-by: Corentin Labbe Signed-off-by: Herbert Xu

crypto: engine - Add KPP Support to Crypto Engine

2021-10-29T13:04:03+00:00

Add KPP support to the crypto engine queue manager, so that it can be used to simplify the logic of KPP device drivers as done for other crypto drivers. Signed-off-by: Prabhjot Khurana Signed-off-by: Daniele Alessandrelli Signed-off-by: Herbert Xu

crypto: engine - delete duplicated word

2020-08-21T04:45:26+00:00

Drop the doubled word "a". Signed-off-by: Randy Dunlap Cc: Herbert Xu Cc: "David S. Miller" Cc: linux-crypto@vger.kernel.org Signed-off-by: Herbert Xu

crypto: algapi - Remove skbuff.h inclusion

2020-08-20T04:04:28+00:00

The header file algapi.h includes skbuff.h unnecessarily since all we need is a forward declaration for struct sk_buff. This patch removes that inclusion. Unfortunately skbuff.h pulls in a lot of things and drivers over the years have come to rely on it so this patch adds a lot of missing inclusions that result from this. Signed-off-by: Herbert Xu

sched,crypto: Convert to sched_set_fifo*()

2020-06-15T12:10:20+00:00

Because SCHED_FIFO is a broken scheduler model (see previous patches) take away the priority field, the kernel can't possibly make an informed decision. Use sched_set_fifo() to request SCHED_FIFO and delegate actual priority selection to userspace. Effectively no change in behaviour. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Ingo Molnar Acked-by: Herbert Xu

crypto: engine - do not requeue in case of fatal error

2020-05-28T07:27:52+00:00

Now, in crypto-engine, if hardware queue is full (-ENOSPC), requeue request regardless of MAY_BACKLOG flag. If hardware throws any other error code (like -EIO, -EINVAL, -ENOMEM, etc.) only MAY_BACKLOG requests are enqueued back into crypto-engine's queue, since the others can be dropped. The latter case can be fatal error, so those cannot be recovered from. For example, in CAAM driver, -EIO is returned in case the job descriptor is broken, so there is no possibility to fix the job descriptor. Therefore, these errors might be fatal error, so we shouldn’t requeue the request. This will just be pass back and forth between crypto-engine and hardware. Fixes: 6a89f492f8e5 ("crypto: engine - support for parallel requests based on retry mechanism") Signed-off-by: Iuliana Prodan Reported-by: Horia Geantă Reviewed-by: Horia Geantă Signed-off-by: Herbert Xu

crypto: engine - support for batch requests

2020-05-08T05:30:40+00:00

Added support for batch requests, per crypto engine. A new callback is added, do_batch_requests, which executes a batch of requests. This has the crypto_engine structure as argument (for cases when more than one crypto-engine is used). The crypto_engine_alloc_init_and_set function, initializes crypto-engine, but also, sets the do_batch_requests callback. On crypto_pump_requests, if do_batch_requests callback is implemented in a driver, this will be executed. The link between the requests will be done in driver, if possible. do_batch_requests is available only if the hardware has support for multiple request. Signed-off-by: Iuliana Prodan Signed-off-by: Herbert Xu