summaryrefslogtreecommitdiff
path: root/crypto/ahash.c
AgeCommit message (Collapse)AuthorFilesLines
2025-06-18crypto: ahash - Fix infinite recursion in ahash_def_finupHerbert Xu1-1/+3
Invoke the final function directly in the default finup implementation since crypto_ahash_final is now just a wrapper around finup. Reported-by: Eric Biggers <ebiggers@kernel.org> Fixes: 9d7a0ab1c753 ("crypto: ahash - Handle partial blocks in API") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-19crypto: hmac - Add ahash supportHerbert Xu1-1/+9
Add ahash support to hmac so that drivers that can't do hmac in hardware do not have to implement duplicate copies of hmac. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-19crypto: shash - Set reqsize in shash_algHerbert Xu1-1/+0
Make reqsize static for shash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-19crypto: ahash - Handle partial blocks in APIHerbert Xu1-279/+248
Provide an option to handle the partial blocks in the ahash API. Almost every hash algorithm has a block size and are only able to hash partial blocks on finalisation. As a first step disable virtual address support for algorithms with state sizes larger than HASH_MAX_STATESIZE. This is OK as virtual addresses are currently only used on synchronous fallbacks. This means ahash_do_req_chain only needs to handle synchronous fallbacks, removing the complexities of saving the request state. Also move the saved request state into the ahash_request object as nesting is no longer possible. Add a scatterlist to ahash_request to store the partial block. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-19crypto: hash - Add export_core and import_core hooksHerbert Xu1-3/+19
Add export_core and import_core hooks. These are intended to be used by algorithms which are wrappers around block-only algorithms, but are not themselves block-only, e.g., hmac. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-05crypto: ahash - Add core export and importHerbert Xu1-1/+24
Add crypto_ahash_export_core and crypto_ahash_import_core. For now they only differ from the normal export/import functions when going through shash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-05crypto: ahash - Enforce MAX_SYNC_HASH_REQSIZE for sync ahashHerbert Xu1-9/+18
As sync ahash algorithms (currently there are none) are used without a fallback, ensure that they obey the MAX_SYNC_HASH_REQSIZE rule just like shash algorithms. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-05crypto: api - Rename CRYPTO_ALG_REQ_CHAIN to CRYPTO_ALG_REQ_VIRTHerbert Xu1-3/+3
As chaining has been removed, all that remains of REQ_CHAIN is just virtual address support. Rename it before the reintroduction of batching creates confusion. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-28crypto: hash - Fix clone error handlingHerbert Xu1-2/+17
Do not copy the exit function in crypto_clone_tfm as it should only be set after init_tfm or clone_tfm has succeeded. Move the setting into crypto_clone_ahash and crypto_clone_shash instead. Also clone the fb if necessary. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-28crypto: api - Add crypto_request_clone and fbHerbert Xu1-27/+5
Add a helper to clone crypto requests and eliminate code duplication. Use kmemdup in the helper. Also add an fb field to crypto_tfm. This also happens to fix the existing implementations which were buggy. Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202504230118.1CxUaUoX-lkp@intel.com/ Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202504230004.c7mrY0C6-lkp@intel.com/ Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-16crypto: hash - Add HASH_REQUEST_ON_STACKHerbert Xu1-4/+102
Allow any ahash to be used with a stack request, with optional dynamic allocation when async is needed. The intended usage is: HASH_REQUEST_ON_STACK(req, tfm); ... err = crypto_ahash_digest(req); /* The request cannot complete synchronously. */ if (err == -EAGAIN) { /* This will not fail. */ req = HASH_REQUEST_CLONE(req, gfp); /* Redo operation. */ err = crypto_ahash_digest(req); } Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-16crypto: api - Add support for duplicating algorithms before registrationHerbert Xu1-0/+1
If the bit CRYPTO_ALG_DUP_FIRST is set, an algorithm will be duplicated by kmemdup before registration. This is inteded for hardware-based algorithms that may be unplugged at will. Do not use this if the algorithm data structure is embedded in a bigger data structure. Perform the duplication in the driver instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-16crypto: ahash - Use cra_reqsizeHerbert Xu1-2/+2
Use the common reqsize field and remove reqsize from ahash_alg. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-16crypto: ahash - Remove request chainingHerbert Xu1-137/+32
Request chaining requires the user to do too much book keeping. Remove it from ahash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-04-12crypto: ahash - Disable request chainingHerbert Xu1-75/+1
Disable hash request chaining in case a driver that copies an ahash_request object by hand accidentally triggers chaining. Reported-by: Manorit Chawdhry <m-chawdhry@ti.com> Fixes: f2ffe5a9183d ("crypto: hash - Add request chaining API") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Tested-by: Manorit Chawdhry <m-chawdhry@ti.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-03-22crypto: hash - Fix synchronous ahash chaining fallbackHerbert Xu1-32/+28
The synchronous ahash fallback code paths are broken because the ahash_restore_req assumes there is always a state object. Fix this by removing the state from ahash_restore_req and localising it to the asynchronous completion callback. Also add a missing synchronous finish call in ahash_def_digest_finish. Fixes: f2ffe5a9183d ("crypto: hash - Add request chaining API") Fixes: 439963cdc3aa ("crypto: ahash - Add virtual address support") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-03-21crypto: hash - Use nth_page instead of doing it by handHerbert Xu1-2/+2
Use nth_page instead of adding n to the page pointer. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-03-21crypto: hash - Fix test underflow in shash_ahash_digestHerbert Xu1-16/+28
The test on PAGE_SIZE - offset in shash_ahash_digest can underflow, leading to execution of the fast path even if the data cannot be mapped into a single page. Fix this by splitting the test into four cases: 1) nbytes > sg->length: More than one SG entry, slow path. 2) !IS_ENABLED(CONFIG_HIGHMEM): fast path. 3) nbytes > (unsigned int)PAGE_SIZE - offset: Two highmem pages, slow path. 4) Highmem fast path. Fixes: 5f7082ed4f48 ("crypto: hash - Export shash through hash") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: ahash - Set default reqsize from ahash_algHerbert Xu1-0/+4
Add a reqsize field to struct ahash_alg and use it to set the default reqsize so that algorithms with a static reqsize are not forced to create an init_tfm function. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: ahash - Add virtual address supportHerbert Xu1-31/+251
This patch adds virtual address support to ahash. Virtual addresses were previously only supported through shash. The user may choose to use virtual addresses with ahash by calling ahash_request_set_virt instead of ahash_request_set_crypt. The API will take care of translating this to an SG list if necessary, unless the algorithm declares that it supports chaining. Therefore in order for an ahash algorithm to support chaining, it must also support virtual addresses directly. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: hash - Add request chaining APIHerbert Xu1-26/+235
This adds request chaining to the ahash interface. Request chaining allows multiple requests to be submitted in one shot. An algorithm can elect to receive chained requests by setting the flag CRYPTO_ALG_REQ_CHAIN. If this bit is not set, the API will break up chained requests and submit them one-by-one. A new err field is added to struct crypto_async_request to record the return value for each individual request. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: ahash - Only save callback and data in ahash_save_reqHerbert Xu1-62/+35
As unaligned operations are supported by the underlying algorithm, ahash_save_req and ahash_restore_req can be greatly simplified to only preserve the callback and data. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: ahash - use str_yes_no() helper in crypto_ahash_show()Thorsten Blum1-2/+3
Remove hard-coded strings by using the str_yes_no() helper function. Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-01-04crypto: ahash - make hash walk functions private to ahash.cEric Biggers1-71/+87
Due to the removal of the Niagara2 SPU driver, crypto_hash_walk_first(), crypto_hash_walk_done(), crypto_hash_walk_last(), and struct crypto_hash_walk are now only used in crypto/ahash.c. Therefore, make them all private to crypto/ahash.c. I.e. un-export the two functions that were exported, make the functions static, and move the struct definition to the .c file. As part of this, move the functions to earlier in the file to avoid needing to add forward declarations. Signed-off-by: Eric Biggers <ebiggers@google.com> Acked-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: remove CONFIG_CRYPTO_STATSEric Biggers1-58/+5
Remove support for the "Crypto usage statistics" feature (CONFIG_CRYPTO_STATS). This feature does not appear to have ever been used, and it is harmful because it significantly reduces performance and is a large maintenance burden. Covering each of these points in detail: 1. Feature is not being used Since these generic crypto statistics are only readable using netlink, it's fairly straightforward to look for programs that use them. I'm unable to find any evidence that any such programs exist. For example, Debian Code Search returns no hits except the kernel header and kernel code itself and translations of the kernel header: https://codesearch.debian.net/search?q=CRYPTOCFGA_STAT&literal=1&perpkg=1 The patch series that added this feature in 2018 (https://lore.kernel.org/linux-crypto/1537351855-16618-1-git-send-email-clabbe@baylibre.com/) said "The goal is to have an ifconfig for crypto device." This doesn't appear to have happened. It's not clear that there is real demand for crypto statistics. Just because the kernel provides other types of statistics such as I/O and networking statistics and some people find those useful does not mean that crypto statistics are useful too. Further evidence that programs are not using CONFIG_CRYPTO_STATS is that it was able to be disabled in RHEL and Fedora as a bug fix (https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2947). Even further evidence comes from the fact that there are and have been bugs in how the stats work, but they were never reported. For example, before Linux v6.7 hash stats were double-counted in most cases. There has also never been any documentation for this feature, so it might be hard to use even if someone wanted to. 2. CONFIG_CRYPTO_STATS significantly reduces performance Enabling CONFIG_CRYPTO_STATS significantly reduces the performance of the crypto API, even if no program ever retrieves the statistics. This primarily affects systems with a large number of CPUs. For example, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2039576 reported that Lustre client encryption performance improved from 21.7GB/s to 48.2GB/s by disabling CONFIG_CRYPTO_STATS. It can be argued that this means that CONFIG_CRYPTO_STATS should be optimized with per-cpu counters similar to many of the networking counters. But no one has done this in 5+ years. This is consistent with the fact that the feature appears to be unused, so there seems to be little interest in improving it as opposed to just disabling it. It can be argued that because CONFIG_CRYPTO_STATS is off by default, performance doesn't matter. But Linux distros tend to error on the side of enabling options. The option is enabled in Ubuntu and Arch Linux, and until recently was enabled in RHEL and Fedora (see above). So, even just having the option available is harmful to users. 3. CONFIG_CRYPTO_STATS is a large maintenance burden There are over 1000 lines of code associated with CONFIG_CRYPTO_STATS, spread among 32 files. It significantly complicates much of the implementation of the crypto API. After the initial submission, many fixes and refactorings have consumed effort of multiple people to keep this feature "working". We should be spending this effort elsewhere. Acked-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-03-13Revert "crypto: remove CONFIG_CRYPTO_STATS"Herbert Xu1-5/+58
This reverts commit 2beb81fbf0c01a62515a1bcef326168494ee2bd0. While removing CONFIG_CRYPTO_STATS is a worthy goal, this also removed unrelated infrastructure such as crypto_comp_alg_common. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-03-01crypto: remove CONFIG_CRYPTO_STATSEric Biggers1-58/+5
Remove support for the "Crypto usage statistics" feature (CONFIG_CRYPTO_STATS). This feature does not appear to have ever been used, and it is harmful because it significantly reduces performance and is a large maintenance burden. Covering each of these points in detail: 1. Feature is not being used Since these generic crypto statistics are only readable using netlink, it's fairly straightforward to look for programs that use them. I'm unable to find any evidence that any such programs exist. For example, Debian Code Search returns no hits except the kernel header and kernel code itself and translations of the kernel header: https://codesearch.debian.net/search?q=CRYPTOCFGA_STAT&literal=1&perpkg=1 The patch series that added this feature in 2018 (https://lore.kernel.org/linux-crypto/1537351855-16618-1-git-send-email-clabbe@baylibre.com/) said "The goal is to have an ifconfig for crypto device." This doesn't appear to have happened. It's not clear that there is real demand for crypto statistics. Just because the kernel provides other types of statistics such as I/O and networking statistics and some people find those useful does not mean that crypto statistics are useful too. Further evidence that programs are not using CONFIG_CRYPTO_STATS is that it was able to be disabled in RHEL and Fedora as a bug fix (https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2947). Even further evidence comes from the fact that there are and have been bugs in how the stats work, but they were never reported. For example, before Linux v6.7 hash stats were double-counted in most cases. There has also never been any documentation for this feature, so it might be hard to use even if someone wanted to. 2. CONFIG_CRYPTO_STATS significantly reduces performance Enabling CONFIG_CRYPTO_STATS significantly reduces the performance of the crypto API, even if no program ever retrieves the statistics. This primarily affects systems with large number of CPUs. For example, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2039576 reported that Lustre client encryption performance improved from 21.7GB/s to 48.2GB/s by disabling CONFIG_CRYPTO_STATS. It can be argued that this means that CONFIG_CRYPTO_STATS should be optimized with per-cpu counters similar to many of the networking counters. But no one has done this in 5+ years. This is consistent with the fact that the feature appears to be unused, so there seems to be little interest in improving it as opposed to just disabling it. It can be argued that because CONFIG_CRYPTO_STATS is off by default, performance doesn't matter. But Linux distros tend to error on the side of enabling options. The option is enabled in Ubuntu and Arch Linux, and until recently was enabled in RHEL and Fedora (see above). So, even just having the option available is harmful to users. 3. CONFIG_CRYPTO_STATS is a large maintenance burden There are over 1000 lines of code associated with CONFIG_CRYPTO_STATS, spread among 32 files. It significantly complicates much of the implementation of the crypto API. After the initial submission, many fixes and refactorings have consumed effort of multiple people to keep this feature "working". We should be spending this effort elsewhere. Cc: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Acked-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-02crypto: ahash - unexport crypto_hash_alg_has_setkey()Eric Biggers1-11/+10
Since crypto_hash_alg_has_setkey() is only called from ahash.c itself, make it a static function. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-11-07crypto: ahash - Set using_shash for cloned ahash wrapper over shashDmitry Safonov1-0/+1
The cloned child of ahash that uses shash under the hood should use shash helpers (like crypto_shash_setkey()). The following panic may be observed on TCP-AO selftests: > ================================================================== > BUG: KASAN: wild-memory-access in crypto_mod_get+0x1b/0x60 > Write of size 4 at addr 5d5be0ff5c415e14 by task connect_ipv4/1397 > > CPU: 0 PID: 1397 Comm: connect_ipv4 Tainted: G W 6.6.0+ #47 > Call Trace: > <TASK> > dump_stack_lvl+0x46/0x70 > kasan_report+0xc3/0xf0 > kasan_check_range+0xec/0x190 > crypto_mod_get+0x1b/0x60 > crypto_spawn_alg+0x53/0x140 > crypto_spawn_tfm2+0x13/0x60 > hmac_init_tfm+0x25/0x60 > crypto_ahash_setkey+0x8b/0x100 > tcp_ao_add_cmd+0xe7a/0x1120 > do_tcp_setsockopt+0x5ed/0x12a0 > do_sock_setsockopt+0x82/0x100 > __sys_setsockopt+0xe9/0x160 > __x64_sys_setsockopt+0x60/0x70 > do_syscall_64+0x3c/0xe0 > entry_SYSCALL_64_after_hwframe+0x46/0x4e > ================================================================== > general protection fault, probably for non-canonical address 0x5d5be0ff5c415e14: 0000 [#1] PREEMPT SMP KASAN > CPU: 0 PID: 1397 Comm: connect_ipv4 Tainted: G B W 6.6.0+ #47 > Call Trace: > <TASK> > ? die_addr+0x3c/0xa0 > ? exc_general_protection+0x144/0x210 > ? asm_exc_general_protection+0x22/0x30 > ? add_taint+0x26/0x90 > ? crypto_mod_get+0x20/0x60 > ? crypto_mod_get+0x1b/0x60 > ? ahash_def_finup_done1+0x58/0x80 > crypto_spawn_alg+0x53/0x140 > crypto_spawn_tfm2+0x13/0x60 > hmac_init_tfm+0x25/0x60 > crypto_ahash_setkey+0x8b/0x100 > tcp_ao_add_cmd+0xe7a/0x1120 > do_tcp_setsockopt+0x5ed/0x12a0 > do_sock_setsockopt+0x82/0x100 > __sys_setsockopt+0xe9/0x160 > __x64_sys_setsockopt+0x60/0x70 > do_syscall_64+0x3c/0xe0 > entry_SYSCALL_64_after_hwframe+0x46/0x4e > </TASK> > RIP: 0010:crypto_mod_get+0x20/0x60 Make sure that the child/clone has using_shash set when parent is an shash user. Fixes: 2f1f34c1bf7b ("crypto: ahash - optimize performance when wrapping shash") Cc: David Ahern <dsahern@kernel.org> Cc: "David S. Miller" <davem@davemloft.net> Cc: Dmitry Safonov <0x7f454c46@gmail.com> Cc: Eric Biggers <ebiggers@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Francesco Ruggeri <fruggeri05@gmail.com> To: Herbert Xu <herbert@gondor.apana.org.au> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Paolo Abeni <pabeni@redhat.com> Cc: Salam Noureddine <noureddine@arista.com> Cc: netdev@vger.kernel.org Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Dmitry Safonov <dima@arista.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27crypto: ahash - optimize performance when wrapping shashEric Biggers1-140/+145
The "ahash" API provides access to both CPU-based and hardware offload- based implementations of hash algorithms. Typically the former are implemented as "shash" algorithms under the hood, while the latter are implemented as "ahash" algorithms. The "ahash" API provides access to both. Various kernel subsystems use the ahash API because they want to support hashing hardware offload without using a separate API for it. Yet, the common case is that a crypto accelerator is not actually being used, and ahash is just wrapping a CPU-based shash algorithm. This patch optimizes the ahash API for that common case by eliminating the extra indirect call for each ahash operation on top of shash. It also fixes the double-counting of crypto stats in this scenario (though CONFIG_CRYPTO_STATS should *not* be enabled by anyone interested in performance anyway...), and it eliminates redundant checking of CRYPTO_TFM_NEED_KEY. As a bonus, it also shrinks struct crypto_ahash. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27crypto: ahash - check for shash type instead of not ahash typeEric Biggers1-5/+3
Since the previous patch made crypto_shash_type visible to ahash.c, change checks for '->cra_type != &crypto_ahash_type' to '->cra_type == &crypto_shash_type'. This makes more sense and avoids having to forward-declare crypto_ahash_type. The result is still the same, since the type is either shash or ahash here. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27crypto: hash - move "ahash wrapping shash" functions to ahash.cEric Biggers1-0/+186
The functions that are involved in implementing the ahash API on top of an shash algorithm belong better in ahash.c, not in shash.c where they currently are. Move them. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27crypto: ahash - improve file commentEric Biggers1-2/+6
Improve the file comment for crypto/ahash.c. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27crypto: ahash - remove struct ahash_request_privEric Biggers1-8/+0
struct ahash_request_priv is unused, so remove it. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27crypto: ahash - remove support for nonzero alignmaskEric Biggers1-109/+8
Currently, the ahash API checks the alignment of all key and result buffers against the algorithm's declared alignmask, and for any unaligned buffers it falls back to manually aligned temporary buffers. This is virtually useless, however. First, since it does not apply to the message, its effect is much more limited than e.g. is the case for the alignmask for "skcipher". Second, the key and result buffers are given as virtual addresses and cannot (in general) be DMA'ed into, so drivers end up having to copy to/from them in software anyway. As a result it's easy to use memcpy() or the unaligned access helpers. The crypto_hash_walk_*() helper functions do use the alignmask to align the message. But with one exception those are only used for shash algorithms being exposed via the ahash API, not for native ahashes, and aligning the message is not required in this case, especially now that alignmask support has been removed from shash. The exception is the n2_core driver, which doesn't set an alignmask. In any case, no ahash algorithms actually set a nonzero alignmask anymore. Therefore, remove support for it from ahash. The benefit is that all the code to handle "misaligned" buffers in the ahash API goes away, reducing the overhead of the ahash API. This follows the same change that was made to shash. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-09-20crypto: hash - Hide CRYPTO_ALG_TYPE_AHASH_MASKHerbert Xu1-0/+2
Move the macro CRYPTO_ALG_TYPE_AHASH_MASK out of linux/crypto.h and into crypto/ahash.c so that it's not visible to users of the Crypto API. Also remove the unused CRYPTO_ALG_TYPE_HASH_MASK macro. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-05-12crypto: hash - Make crypto_ahash_alg helper availableHerbert Xu1-6/+0
Move the crypto_ahash_alg helper into include/crypto/internal so that drivers can use it. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-05-12crypto: hash - Add statesize to crypto_ahashHerbert Xu1-0/+3
As ahash drivers may need to use fallbacks, their state size is thus variable. Deal with this by making it an attribute of crypto_ahash. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-05-02crypto: api - Fix CRYPTO_USER checks for report functionOndrej Mosnacek1-1/+1
Checking the config via ifdef incorrectly compiles out the report functions when CRYPTO_USER is set to =m. Fix it by using IS_ENABLED() instead. Fixes: c0f9e01dd266 ("crypto: api - Check CRYPTO_USER instead of NET for report") Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-20crypto: hash - Add crypto_clone_ahash/shashHerbert Xu1-0/+51
This patch adds the helpers crypto_clone_ahash and crypto_clone_shash. They are the hash-specific counterparts of crypto_clone_tfm. This allows code paths that cannot otherwise allocate a hash tfm object to do so. Once a new tfm has been obtained its key could then be changed without impacting other users. Note that only algorithms that implement clone_tfm can be cloned. However, all keyless hashes can be cloned by simply reusing the tfm object. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Simon Horman <simon.horman@corigine.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-03-14crypto: api - Check CRYPTO_USER instead of NET for reportHerbert Xu1-8/+4
The report function is currently conditionalised on CONFIG_NET. As it's only used by CONFIG_CRYPTO_USER, conditionalising on that instead of CONFIG_NET makes more sense. This gets rid of a rarely used code-path. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-03-14crypto: hash - Count error stats differentlyHerbert Xu1-33/+48
Move all stat code specific to hash into the hash code. While we're at it, change the stats so that bytes and counts are always incremented even in case of error. This allows the reference counting to be removed as we can now increment the counters prior to the operation. After the operation we simply increase the error count if necessary. This is safe as errors can only occur synchronously (or rather, the existing code already ignored asynchronous errors which are only visible to the callback function). Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-13crypto: api - Use data directly in completion functionHerbert Xu1-6/+6
This patch does the final flag day conversion of all completion functions which are now all contained in the Crypto API. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-02-13crypto: hash - Use crypto_request_completeHerbert Xu1-105/+74
Use the crypto_request_complete helper instead of calling the completion function directly. This patch also removes the voodoo programming previously used for unaligned ahash operations and replaces it with a sub-request. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-12-30crypto: scatterwalk - use kmap_local() not kmap_atomic()Ard Biesheuvel1-2/+2
kmap_atomic() is used to create short-lived mappings of pages that may not be accessible via the kernel direct map. This is only needed on 32-bit architectures that implement CONFIG_HIGHMEM, but it can be used on 64-bit other architectures too, where the returned mapping is simply the kernel direct address of the page. However, kmap_atomic() does not support migration on CONFIG_HIGHMEM configurations, due to the use of per-CPU kmap slots, and so it disables preemption on all architectures, not just the 32-bit ones. This implies that all scatterwalk based crypto routines essentially execute with preemption disabled all the time, which is less than ideal. So let's switch scatterwalk_map/_unmap and the shash/ahash routines to kmap_local() instead, which serves a similar purpose, but without the resulting impact on preemption on architectures that have no need for CONFIG_HIGHMEM. Cc: Eric Biggers <ebiggers@kernel.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "Elliott, Robert (Servers)" <elliott@hpe.com> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-08-28crypto: ahash - Add init_tfm/exit_tfmHerbert Xu1-1/+12
This patch adds the type-safe init_tfm/exit_tfm functions to the ahash interface. This is meant to replace the unsafe cra_init and cra_exit interface. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-08-21crypto: hash - Remove unused async iteratorsIra Weiny1-37/+4
Revert "crypto: hash - Add real ahash walk interface" This reverts commit 75ecb231ff45b54afa9f4ec9137965c3c00868f4. The callers of the functions in this commit were removed in ab8085c130ed Remove these unused calls. Fixes: ab8085c130ed ("crypto: x86 - remove SHA multibuffer routines and mcryptd") Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Ira Weiny <ira.weiny@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-08-07mm, treewide: rename kzfree() to kfree_sensitive()Waiman Long1-2/+2
As said by Linus: A symmetric naming is only helpful if it implies symmetries in use. Otherwise it's actively misleading. In "kzalloc()", the z is meaningful and an important part of what the caller wants. In "kzfree()", the z is actively detrimental, because maybe in the future we really _might_ want to use that "memfill(0xdeadbeef)" or something. The "zero" part of the interface isn't even _relevant_. The main reason that kzfree() exists is to clear sensitive information that should not be leaked to other future users of the same memory objects. Rename kzfree() to kfree_sensitive() to follow the example of the recently added kvfree_sensitive() and make the intention of the API more explicit. In addition, memzero_explicit() is used to clear the memory to make sure that it won't get optimized away by the compiler. The renaming is done by using the command sequence: git grep -w --name-only kzfree |\ xargs sed -i 's/kzfree/kfree_sensitive/' followed by some editing of the kfree_sensitive() kerneldoc and adding a kzfree backward compatibility macro in slab.h. [akpm@linux-foundation.org: fs/crypto/inline_crypt.c needs linux/slab.h] [akpm@linux-foundation.org: fix fs/crypto/inline_crypt.c some more] Suggested-by: Joe Perches <joe@perches.com> Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Howells <dhowells@redhat.com> Acked-by: Michal Hocko <mhocko@suse.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> Cc: James Morris <jmorris@namei.org> Cc: "Serge E. Hallyn" <serge@hallyn.com> Cc: Joe Perches <joe@perches.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: David Rientjes <rientjes@google.com> Cc: Dan Carpenter <dan.carpenter@oracle.com> Cc: "Jason A . Donenfeld" <Jason@zx2c4.com> Link: http://lkml.kernel.org/r/20200616154311.12314-3-longman@redhat.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-01-09crypto: algapi - enforce that all instances have a ->free() methodEric Biggers1-0/+3
All instances need to have a ->free() method, but people could forget to set it and then not notice if the instance is never unregistered. To help detect this bug earlier, don't allow an instance without a ->free() method to be registered, and complain loudly if someone tries to do it. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-01-09crypto: algapi - remove crypto_template::{alloc,free}()Eric Biggers1-5/+0
Now that all templates provide a ->create() method which creates an instance, installs a strongly-typed ->free() method directly to it, and registers it, the older ->alloc() and ->free() methods in 'struct crypto_template' are no longer used. Remove them. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>