summaryrefslogtreecommitdiff
path: root/drivers/crypto/intel/qat/qat_common
AgeCommit message (Collapse)AuthorFilesLines
2024-11-10crypto: qat - Fix missing destroy_workqueue in adf_init_aer()Wang Hai1-1/+4
The adf_init_aer() won't destroy device_reset_wq when alloc_workqueue() for device_sriov_wq failed. Add destroy_workqueue for device_reset_wq to fix this issue. Fixes: 4469f9b23468 ("crypto: qat - re-enable sriov after pf reset") Signed-off-by: Wang Hai <wanghai38@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-10-28crypto: qat - Fix typo "accelaration"WangYuli1-1/+1
There is a spelling mistake of 'accelaration' in comments which should be 'acceleration'. Signed-off-by: WangYuli <wangyuli@uniontech.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-10-28crypto: qat - Constify struct pm_status_rowChristophe JAILLET1-9/+9
'struct pm_status_row' are not modified in this driver. Constifying this structure moves some data to a read-only section, so increases overall security. Update the prototype of some functions accordingly. On a x86_64, with allmodconfig, as an example: Before: ====== text data bss dec hex filename 4400 1059 0 5459 1553 drivers/crypto/intel/qat/qat_common/adf_gen4_pm_debugfs.o After: ===== text data bss dec hex filename 5216 243 0 5459 1553 drivers/crypto/intel/qat/qat_common/adf_gen4_pm_debugfs.o Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-10-19crypto: qat - remove faulty arbiter config resetAhsan Atta1-4/+0
Resetting the service arbiter config can cause potential issues related to response ordering and ring flow control check in the event of AER or device hang. This is because it results in changing the default response ring size from 32 bytes to 16 bytes. The service arbiter config reset also disables response ring flow control check. Thus, by removing this reset we can prevent the service arbiter from being configured inappropriately, which leads to undesired device behaviour in the event of errors. Fixes: 7afa232e76ce ("crypto: qat - Intel(R) QAT DH895xcc accelerator") Signed-off-by: Ahsan Atta <ahsan.atta@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-10-05crypto: qat - remove unused adf_devmgr_get_firstDr. David Alan Gilbert2-11/+0
adf_devmgr_get_first has been unused since commit 4a4b0bad0653 ("crypto: qat - fix crypto_get_instance_node function") Remove it. (Build tested only, I don't own the hardware) Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org> Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-10-05crypto: qat - remove check after debugfs_create_dir()Cabiddu, Giovanni1-12/+1
The debugfs functions are guaranteed to return a valid error code instead of NULL upon failure. Consequently, the driver can directly propagate any error returned without additional checks. Remove the unnecessary `if` statement after debugfs_create_dir(). If this function fails, the error code is stored in accel_dev->debugfs_dir and utilized in subsequent debugfs calls. Additionally, since accel_dev->debugfs_dir is assured to be non-NULL, remove the superfluous NULL pointer checks within the adf_dbgfs_add() and adf_dbgfs_rm(). Fixes: 9260db6640a6 ("crypto: qat - move dbgfs init to separate file") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-09-06crypto: qat - Remove trailing space after \n newlineColin Ian King1-1/+1
There is a extraneous space after a newline in a pr_err message. Remove it. Signed-off-by: Colin Ian King <colin.i.king@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-24crypto: qat - fix "Full Going True" macro definitionSvyatoslav Pankratov1-1/+1
The macro `ADF_RP_INT_SRC_SEL_F_RISE_MASK` is currently set to the value `0100b` which means "Empty Going False". This might cause an incorrect restore of the bank state during live migration. Fix the definition of the macro to properly represent the "Full Going True" state which is encoded as `0011b`. Fixes: bbfdde7d195f ("crypto: qat - add bank save and restore flows") Signed-off-by: Svyatoslav Pankratov <svyatoslav.pankratov@intel.com> Reviewed-by: Xin Zeng <xin.zeng@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-17crypto: qat - Use static_assert() to check struct sizesGustavo A. R. Silva1-0/+2
Commit 140e4c85d540 ("crypto: qat - Avoid -Wflex-array-member-not-at-end warnings") introduced tagged `struct qat_alg_buf_list_hdr`. We want to ensure that when new members need to be added to the flexible structure, they are always included within this tagged struct. So, we use `static_assert()` to ensure that the memory layout for both the flexible structure and the tagged struct is the same after any changes. Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-02crypto: qat - allow disabling SR-IOV VFsMichal Witwicki1-66/+128
The QAT driver allows enabling SR-IOV VFs but does not allow them to be disabled through a write to sysfs. Disabling SR-IOV VFs can be only achieved by bringing down and up a device using the attribute /sys/bus/pci/devices/<BDF>/qat/state. The documentation for the sysfs attribute `sriov_numvfs` specifies that "a userspace application wanting to disable the VFs would write a zero to this file". Add support for disabling SR-IOV VFs by writing '0' to the 'sriov_numvfs' attribute in sysfs. Enabling or disabling SR-IOV always requires adf_dev_down() to be called. This action subsequently leads to the deletion of the ADF_KERNEL_SEC configuration section. The keys ADF_NUM_CY and ADF_NUM_DC within that section must be set to '0', otherwise, the driver will register into the Linux Crypto Framework. Because of this, the configuration in the ADF_KERNEL_SEC section must be added before every sriov_enable. Signed-off-by: Michal Witwicki <michal.witwicki@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-02crypto: qat - ensure correct order in VF restarting handlerMichal Witwicki1-3/+6
In the process of sending the ADF_PF2VF_MSGTYPE_RESTARTING message to Virtual Functions (VFs), the Physical Function (PF) should set the `vf->restarting` flag to true before dispatching the message. This change is necessary to prevent a race condition where the handling of the ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE message (which sets the `vf->restarting` flag to false) runs immediately after the message is sent, but before the flag is set to true. Set the `vf->restarting` to true before sending the message ADF_PF2VF_MSGTYPE_RESTARTING, if supported by the version of the protocol and if the VF is started. Fixes: ec26f8e6c784 ("crypto: qat - update PFVF protocol for recovery") Signed-off-by: Michal Witwicki <michal.witwicki@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-02crypto: qat - fix recovery flow for VFsMichal Witwicki3-0/+17
When the PFVF protocol was updated to support version 5, i.e. ADF_PFVF_COMPAT_FALLBACK, the compatibility version for the VF was updated without supporting the message RESTARTING_COMPLETE required for such version. Add support for the ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE message in the VF drivers. This message is sent by the VF driver to the PF to notify the completion of the shutdown flow. Fixes: ec26f8e6c784 ("crypto: qat - update PFVF protocol for recovery") Signed-off-by: Michal Witwicki <michal.witwicki@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-02crypto: qat - disable IOV in adf_dev_stop()Michal Witwicki1-2/+2
Disabling IOV has the side effect of re-enabling the AEs that might attempt to do DMAs into the heartbeat buffers. Move the disable_iov() function in adf_dev_stop() before the AEs are stopped. Fixes: ed8ccaef52fa ("crypto: qat - Add support for SRIOV") Signed-off-by: Michal Witwicki <michal.witwicki@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-08-02crypto: qat - preserve ADF_GENERAL_SECAdam Guerin9-45/+44
The ADF_GENERAL_SEC configuration section contains values that must be preserved during state transitions (down -> up, up -> down). This patch modifies the logic in adf_dev_shutdown() to maintain all key values within this section, rather than selectively saving and restoring only the ADF_SERVICES_ENABLED attribute. To achieve this, a new function has been introduced that deletes all configuration sections except for the one specified by name. This function is invoked during adf_dev_down(), with ADF_GENERAL_SEC as the argument. Consequently, the adf_dev_shutdown_cache_cfg() function has been removed as it is now redundant. Additionally, this patch eliminates the cache_config parameter from the adf_dev_down() function since ADF_GENERAL_SEC should always be retained. This change does not cause any side effects because all entries in the key-value store are cleared when a module is unloaded. Signed-off-by: Adam Guerin <adam.guerin@intel.com> Co-developed-by: Michal Witwicki <michal.witwicki@intel.com> Signed-off-by: Michal Witwicki <michal.witwicki@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-07-19Merge tag 'v6.11-p1' of ↵Linus Torvalds5-13/+21
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto update from Herbert Xu: "API: - Test setkey in no-SIMD context - Add skcipher speed test for user-specified algorithm Algorithms: - Add x25519 support on ppc64le - Add VAES and AVX512 / AVX10 optimized AES-GCM on x86 - Remove sm2 algorithm Drivers: - Add Allwinner H616 support to sun8i-ce - Use DMA in stm32 - Add Exynos850 hwrng support to exynos" * tag 'v6.11-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (81 commits) hwrng: core - remove (un)register_miscdev() crypto: lib/mpi - delete unnecessary condition crypto: testmgr - generate power-of-2 lengths more often crypto: mxs-dcp - Ensure payload is zero when using key slot hwrng: Kconfig - Do not enable by default CN10K driver crypto: starfive - Fix nent assignment in rsa dec crypto: starfive - Align rsa input data to 32-bit crypto: qat - fix unintentional re-enabling of error interrupts crypto: qat - extend scope of lock in adf_cfg_add_key_value_param() Documentation: qat: fix auto_reset attribute details crypto: sun8i-ce - add Allwinner H616 support crypto: sun8i-ce - wrap accesses to descriptor address fields dt-bindings: crypto: sun8i-ce: Add compatible for H616 hwrng: core - Fix wrong quality calculation at hw rng registration hwrng: exynos - Enable Exynos850 support hwrng: exynos - Add SMC based TRNG operation hwrng: exynos - Implement bus clock control hwrng: exynos - Use devm_clk_get_enabled() to get the clock hwrng: exynos - Improve coding style dt-bindings: rng: Add Exynos850 support to exynos-trng ...
2024-07-06crypto: qat - fix unintentional re-enabling of error interruptsHareshx Sankar Raj1-1/+3
The logic that detects pending VF2PF interrupts unintentionally clears the section of the error mask register(s) not related to VF2PF. This might cause interrupts unrelated to VF2PF, reported through errsou3 and errsou5, to be reported again after the execution of the function disable_pending_vf2pf_interrupts() in dh895xcc and GEN2 devices. Fix by updating only section of errmsk3 and errmsk5 related to VF2PF. Signed-off-by: Hareshx Sankar Raj <hareshx.sankar.raj@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-07-06crypto: qat - extend scope of lock in adf_cfg_add_key_value_param()Nivas Varadharajan Mugunthakumar1-2/+4
The function adf_cfg_add_key_value_param() attempts to access and modify the key value store of the driver without locking. Extend the scope of cfg->lock to avoid a potential race condition. Fixes: 92bf269fbfe9 ("crypto: qat - change behaviour of adf_cfg_add_key_value_param()") Signed-off-by: Nivas Varadharajan Mugunthakumar <nivasx.varadharajan.mugunthakumar@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-06-28crypto: qat - initialize user_input.lock for rate_limitingJiwei Sun1-0/+1
If the following configurations are set, CONFIG_DEBUG_RWSEMS=y CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_RWSEM_SPIN_ON_OWNER=y And run the following command, [root@localhost sys]# cat /sys/devices/pci0000:6b/0000:6b:00.0/qat_rl/pir The following warning log appears, ------------[ cut here ]------------ DEBUG_RWSEMS_WARN_ON(sem->magic != sem): count = 0x0, magic = 0x0, owner = 0x1, curr 0xff11000119288040, list not empty WARNING: CPU: 131 PID: 1254984 at kernel/locking/rwsem.c:1280 down_read+0x439/0x7f0 CPU: 131 PID: 1254984 Comm: cat Kdump: loaded Tainted: G W 6.10.0-rc4+ #86 b2ae60c8ceabed15f4fd2dba03c1c5a5f7f4040c Hardware name: Lenovo ThinkServer SR660 V3/SR660 V3, BIOS T8E166X-2.54 05/30/2024 RIP: 0010:down_read+0x439/0x7f0 Code: 44 24 10 80 3c 02 00 0f 85 05 03 00 00 48 8b 13 41 54 48 c7 c6 a0 3e 0e b4 48 c7 c7 e0 3e 0e b4 4c 8b 4c 24 08 e8 77 d5 40 fd <0f> 0b 59 e9 bc fc ff ff 0f 1f 44 00 00 e9 e2 fd ff ff 4c 8d 7b 08 RSP: 0018:ffa0000035f67a78 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ff1100012b03a658 RCX: 0000000000000000 RDX: 0000000080000002 RSI: 0000000000000008 RDI: 0000000000000001 RBP: 1ff4000006becf53 R08: fff3fc0006becf17 R09: fff3fc0006becf17 R10: fff3fc0006becf16 R11: ffa0000035f678b7 R12: ffffffffb40e3e60 R13: ffffffffb627d1f4 R14: ff1100012b03a6d0 R15: ff1100012b03a6c8 FS: 00007fa9ff9a6740(0000) GS:ff1100081e600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fa9ff984000 CR3: 00000002118ae006 CR4: 0000000000771ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: <TASK> pir_show+0x5d/0xe0 [intel_qat 9e297e249ab040329cf58b657b06f418fd5c5855] dev_attr_show+0x3f/0xc0 sysfs_kf_seq_show+0x1ce/0x400 seq_read_iter+0x3fa/0x10b0 vfs_read+0x6f5/0xb20 ksys_read+0xe9/0x1d0 do_syscall_64+0x8a/0x170 entry_SYSCALL_64_after_hwframe+0x76/0x7e RIP: 0033:0x7fa9ff6fd9b2 Code: c0 e9 b2 fe ff ff 50 48 8d 3d ea 1d 0c 00 e8 c5 fd 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24 RSP: 002b:00007ffc0616b968 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007fa9ff6fd9b2 RDX: 0000000000020000 RSI: 00007fa9ff985000 RDI: 0000000000000003 RBP: 00007fa9ff985000 R08: 00007fa9ff984010 R09: 0000000000000000 R10: 0000000000000022 R11: 0000000000000246 R12: 0000000000022000 R13: 0000000000000003 R14: 0000000000020000 R15: 0000000000020000 </TASK> irq event stamp: 0 hardirqs last enabled at (0): [<0000000000000000>] 0x0 hardirqs last disabled at (0): [<ffffffffb102c126>] copy_process+0x21e6/0x6e70 softirqs last enabled at (0): [<ffffffffb102c176>] copy_process+0x2236/0x6e70 softirqs last disabled at (0): [<0000000000000000>] 0x0 ---[ end trace 0000000000000000 ]--- The rate_limiting->user_input.lock rwsem lock is not initialized before use. Let's initialize it. Signed-off-by: Jiwei Sun <sunjw10@lenovo.com> Reviewed-by: Adrian Huang <ahuang12@lenovo.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-06-21crypto: qat - make adf_ctl_class constantGreg Kroah-Hartman1-9/+12
Now that the driver core allows for struct class to be in read-only memory, we should make all 'class' structures declared at build time placing them into read-only memory, instead of having to be dynamically allocated at runtime. Cc: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: Adam Guerin <adam.guerin@intel.com> Cc: Benjamin Tissoires <benjamin.tissoires@redhat.com> Cc: Tom Zanussi <tom.zanussi@linux.intel.com> Cc: Shashank Gupta <shashank.gupta@intel.com> Cc: qat-linux@intel.com Cc: linux-crypto@vger.kernel.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-06-21crypto: qat - fix linking errors when PCI_IOV is disabledXin Zeng1-2/+3
When CONFIG_PCI_IOV=n, the build of the QAT vfio pci variant driver fails reporting the following linking errors: ERROR: modpost: "qat_vfmig_open" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_resume" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_save_state" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_suspend" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_load_state" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_reset" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_save_setup" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_destroy" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_close" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! ERROR: modpost: "qat_vfmig_cleanup" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined! WARNING: modpost: suppressed 1 unresolved symbol warnings because there were too many) Make live migration helpers provided by QAT PF driver always available even if CONFIG_PCI_IOV is not selected. This does not cause any side effect. Reported-by: Arnd Bergmann <arnd@arndb.de> Closes: https://lore.kernel.org/lkml/20240607153406.60355e6c.alex.williamson@redhat.com/T/ Fixes: bb208810b1ab ("vfio/qat: Add vfio_pci driver for Intel QAT SR-IOV VF devices") Signed-off-by: Xin Zeng <xin.zeng@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-05-31crypto: qat - Fix typoDeming Wang1-1/+1
The mapings should be replaced by mappings. Signed-off-by: Deming Wang <wangdeming@inspur.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-05-17crypto: qat - Fix ADF_DEV_RESET_SYNC memory leakHerbert Xu1-14/+5
Using completion_done to determine whether the caller has gone away only works after a complete call. Furthermore it's still possible that the caller has not yet called wait_for_completion, resulting in another potential UAF. Fix this by making the caller use cancel_work_sync and then freeing the memory safely. Fixes: 7d42e097607c ("crypto: qat - resolve race condition during AER recovery") Cc: <stable@vger.kernel.org> #6.8+ Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-26crypto: qat - validate slices count returned by FWLucas Segarra Fernandez3-0/+23
The function adf_send_admin_tl_start() enables the telemetry (TL) feature on a QAT device by sending the ICP_QAT_FW_TL_START message to the firmware. This triggers the FW to start writing TL data to a DMA buffer in memory and returns an array containing the number of accelerators of each type (slices) supported by this HW. The pointer to this array is stored in the adf_tl_hw_data data structure called slice_cnt. The array slice_cnt is then used in the function tl_print_dev_data() to report in debugfs only statistics about the supported accelerators. An incorrect value of the elements in slice_cnt might lead to an out of bounds memory read. At the moment, there isn't an implementation of FW that returns a wrong value, but for robustness validate the slice count array returned by FW. Fixes: 69e7649f7cc2 ("crypto: qat - add support for device telemetry") Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-19crypto: qat - improve error logging to be consistent across featuresAdam Guerin1-1/+1
Improve error logging in rate limiting feature. Staying consistent with the error logging found in the telemetry feature. Fixes: d9fb8408376e ("crypto: qat - add rate limiting feature to qat_4xxx") Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-19crypto: qat - implement dh fallback for primes > 4KDamian Muszynski1-6/+60
The Intel QAT driver provides support for the Diffie-Hellman (DH) algorithm, limited to prime numbers up to 4K. This driver is used by default on platforms with integrated QAT hardware for all DH requests. This has led to failures with algorithms requiring larger prime sizes, such as ffdhe6144. alg: ffdhe6144(dh): test failed on vector 1, err=-22 alg: self-tests for ffdhe6144(qat-dh) (ffdhe6144(dh)) failed (rc=-22) Implement a fallback mechanism when an unsupported request is received. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-12crypto: qat - Fix spelling mistake "Invalide" -> "Invalid"Colin Ian King1-1/+1
There is a spelling mistake in a dev_err message. Fix it. Signed-off-by: Colin Ian King <colin.i.king@gmail.com> Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-05crypto: qat - Avoid -Wflex-array-member-not-at-end warningsGustavo A. R. Silva2-6/+11
-Wflex-array-member-not-at-end is coming in GCC-14, and we are getting ready to enable it globally. Use the `__struct_group()` helper to separate the flexible array from the rest of the members in flexible `struct qat_alg_buf_list`, through tagged `struct qat_alg_buf_list_hdr`, and avoid embedding the flexible-array member in the middle of `struct qat_alg_fixed_buf_list`. Also, use `container_of()` whenever we need to retrieve a pointer to the flexible structure. So, with these changes, fix the following warnings: drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] drivers/crypto/intel/qat/qat_common/qat_bl.h:25:33: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end] Link: https://github.com/KSPP/linux/issues/202 Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org> Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - implement interface for live migrationXin Zeng7-1/+1441
Add logic to implement the interface for live migration defined in qat/qat_mig_dev.h. This is specific for QAT GEN4 Virtual Functions (VFs). This introduces a migration data manager which is used to handle the device state during migration. The manager ensures that the device state is stored in a format that can be restored in the destination node. The VF state is organized into a hierarchical structure that includes a preamble, a general state section, a MISC bar section and an ETR bar section. The latter contains the state of the 4 ring pairs contained on a VF. Here is a graphical representation of the state: preamble | general state section | leaf state | MISC bar state section| leaf state | ETR bar state section | bank0 state section | leaf state | bank1 state section | leaf state | bank2 state section | leaf state | bank3 state section | leaf state In addition to the implementation of the qat_migdev_ops interface and the state manager framework, add a mutex in pfvf to avoid pf2vf messages during migration. Signed-off-by: Xin Zeng <xin.zeng@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - add interface for live migrationXin Zeng4-1/+158
Extend the driver with a new interface to be used for VF live migration. This allows to create and destroy a qat_mig_dev object that contains a set of methods to allow to save and restore the state of QAT VF. This interface will be used by the qat-vfio-pci module. Signed-off-by: Xin Zeng <xin.zeng@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - add bank save and restore flowsSiming Wan3-0/+336
Add logic to save, restore, quiesce and drain a ring bank for QAT GEN4 devices. This allows to save and restore the state of a Virtual Function (VF) and will be used to implement VM live migration. Signed-off-by: Siming Wan <siming.wan@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Xin Zeng <xin.zeng@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - expand CSR operations for QAT GEN4 devicesSiming Wan3-1/+249
Extend the CSR operations for QAT GEN4 devices to allow saving and restoring the rings state. The new operations will be used as a building block for implementing the state save and restore of Virtual Functions necessary for VM live migration. This adds the following operations: - read ring status register - read ring underflow/overflow status register - read ring nearly empty status register - read ring nearly full status register - read ring full status register - read ring complete status register - read ring exception status register - read/write ring exception interrupt mask register - read ring configuration register - read ring base register - read/write ring interrupt enable register - read ring interrupt flag register - read/write ring interrupt source select register - read ring coalesced interrupt enable register - read ring coalesced interrupt control register - read ring flag and coalesced interrupt enable register - read ring service arbiter enable register - get ring coalesced interrupt control enable mask Signed-off-by: Siming Wan <siming.wan@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Xin Zeng <xin.zeng@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - rename get_sla_arr_of_type()Siming Wan2-5/+7
The function get_sla_arr_of_type() returns a pointer to an SLA type specific array. Rename it and expose it as it will be used externally to this module. This does not introduce any functional change. Signed-off-by: Siming Wan <siming.wan@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Signed-off-by: Xin Zeng <xin.zeng@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - relocate CSR access codeGiovanni Cabiddu9-362/+389
As the common hw_data files are growing and the adf_hw_csr_ops is going to be extended with new operations, move all logic related to ring CSRs to the newly created adf_gen[2|4]_hw_csr_data.[c|h] files. This does not introduce any functional change. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Xin Zeng <xin.zeng@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - move PFVF compat checker to a functionXin Zeng2-7/+12
Move the code that implements VF version compatibility on the PF side to a separate function so that it can be reused when doing VM live migration. This does not introduce any functional change. Signed-off-by: Xin Zeng <xin.zeng@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - relocate and rename 4xxx PF2VM definitionsXin Zeng2-5/+7
Move and rename ADF_4XXX_PF2VM_OFFSET and ADF_4XXX_VM2PF_OFFSET to ADF_GEN4_PF2VM_OFFSET and ADF_GEN4_VM2PF_OFFSET respectively. These definitions are moved from adf_gen4_pfvf.c to adf_gen4_hw_data.h as they are specific to GEN4 and not just to qat_4xxx. This change is made in anticipation of their use in live migration. This does not introduce any functional change. Signed-off-by: Xin Zeng <xin.zeng@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02crypto: qat - adf_get_etr_base() helperGiovanni Cabiddu3-6/+12
Add and use the new helper function adf_get_etr_base() which retrieves the virtual address of the ring bar. This will be used extensively when adding support for Live Migration. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Xin Zeng <xin.zeng@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - make ring to service map common for QAT GEN4Damian Muszynski3-0/+58
The function get_ring_to_svc_map() is present in both 420xx and 4xxx drivers. Rework the logic to make it generic to GEN4 devices and move it to qat_common/adf_gen4_hw_data.c. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - fix comment structureAdam Guerin1-2/+2
Move comment description to the same line as the function name. This is to fix the following warning when compiling the QAT driver using the clang compiler with CC=clang W=2: drivers/crypto/intel/qat/qat_common/qat_crypto.c:108: warning: missing initial short description on line: * qat_crypto_vf_dev_config() Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - remove unnecessary description from commentAdam Guerin3-6/+2
Remove extra description from comments as it is not required. This is to fix the following warning when compiling the QAT driver using the clang compiler with CC=clang W=2: drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c:65: warning: contents before sections drivers/crypto/intel/qat/qat_common/adf_isr.c:380: warning: contents before sections drivers/crypto/intel/qat/qat_common/adf_vf_isr.c:298: warning: contents before sections Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - remove double initialization of valueAdam Guerin1-4/+2
Remove double initialization of the reg variable. This is to fix the following warning when compiling the QAT driver using clang scan-build: drivers/crypto/intel/qat/qat_common/adf_gen4_ras.c:1010:6: warning: Value stored to 'reg' during its initialization is never read [deadcode.DeadStores] 1010 | u32 reg = ADF_CSR_RD(csr, ADF_GEN4_SSMCPPERR); | ^~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/crypto/intel/qat/qat_common/adf_gen4_ras.c:1109:6: warning: Value stored to 'reg' during its initialization is never read [deadcode.DeadStores] 1109 | u32 reg = ADF_CSR_RD(csr, ADF_GEN4_SER_ERR_SSMSH); | ^~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Fixes: 99b1c9826e48 ("crypto: qat - count QAT GEN4 errors") Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - avoid division by zeroAdam Guerin1-0/+3
Check if delta_us is not zero and return -EINVAL if it is. delta_us is unlikely to be zero as there is a sleep between the reads of the two timestamps. This is to fix the following warning when compiling the QAT driver using clang scan-build: drivers/crypto/intel/qat/qat_common/adf_clock.c:87:9: warning: Division by zero [core.DivideZero] 87 | temp = DIV_ROUND_CLOSEST_ULL(temp, delta_us); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Fixes: e2980ba57e79 ("crypto: qat - add measure clock frequency") Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - removed unused macro in adf_cnv_dbgfs.cAdam Guerin1-1/+0
This macro was added but never used, remove it. This is to fix the following warning when compiling the QAT driver using the clang compiler with CC=clang W=2: drivers/crypto/intel/qat/qat_common/adf_cnv_dbgfs.c:19:9: warning: macro is not used [-Wunused-macros] 19 | #define CNV_SLICE_ERR_MASK GENMASK(7, 0) | ^ Fixes: d807f0240c71 ("crypto: qat - add cnv_errors debugfs file") Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - remove unused macros in qat_comp_alg.cAdam Guerin1-9/+0
As a result of the removal of qat_zlib_deflate, some defines where not removed. Remove them. This is to fix the following warning when compiling the QAT driver using the clang compiler with CC=clang W=2: drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:21:9: warning: macro is not used [-Wunused-macros] 21 | #define QAT_RFC_1950_CM_OFFSET 4 | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:16:9: warning: macro is not used [-Wunused-macros] 16 | #define QAT_RFC_1950_HDR_SIZE 2 | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:17:9: warning: macro is not used [-Wunused-macros] 17 | #define QAT_RFC_1950_FOOTER_SIZE 4 | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:22:9: warning: macro is not used [-Wunused-macros] 22 | #define QAT_RFC_1950_DICT_MASK 0x20 | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:18:9: warning: macro is not used [-Wunused-macros] 18 | #define QAT_RFC_1950_CM_DEFLATE 8 | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:20:9: warning: macro is not used [-Wunused-macros] 20 | #define QAT_RFC_1950_CM_MASK 0x0f | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:23:9: warning: macro is not used [-Wunused-macros] 23 | #define QAT_RFC_1950_COMP_HDR 0x785e | ^ drivers/crypto/intel/qat/qat_common/qat_comp_algs.c:19:9: warning: macro is not used [-Wunused-macros] 19 | #define QAT_RFC_1950_CM_DEFLATE_CINFO_32K 7 | ^ Fixes: e9dd20e0e5f6 ("crypto: qat - Remove zlib-deflate") Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-24crypto: qat - uninitialized variable in adf_hb_error_inject_write()Dan Carpenter1-5/+6
There are a few issues in this code. If *ppos is non-zero then the first part of the buffer is not initialized. We never initialize the last character of the buffer. The return is not checked so it's possible that none of the buffer is initialized. This is debugfs code which is root only and the impact of these bugs is very small. However, it's still worth fixing. To fix this: 1) Check that *ppos is zero. 2) Use copy_from_user() instead of simple_write_to_buffer(). 3) Explicitly add a NUL terminator. Fixes: e2b67859ab6e ("crypto: qat - add heartbeat error simulator") Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-17crypto: qat - resolve race condition during AER recoveryDamian Muszynski1-6/+16
During the PCI AER system's error recovery process, the kernel driver may encounter a race condition with freeing the reset_data structure's memory. If the device restart will take more than 10 seconds the function scheduling that restart will exit due to a timeout, and the reset_data structure will be freed. However, this data structure is used for completion notification after the restart is completed, which leads to a UAF bug. This results in a KFENCE bug notice. BUG: KFENCE: use-after-free read in adf_device_reset_worker+0x38/0xa0 [intel_qat] Use-after-free read at 0x00000000bc56fddf (in kfence-#142): adf_device_reset_worker+0x38/0xa0 [intel_qat] process_one_work+0x173/0x340 To resolve this race condition, the memory associated to the container of the work_struct is freed on the worker if the timeout expired, otherwise on the function that schedules the worker. The timeout detection can be done by checking if the caller is still waiting for completion or not by using completion_done() function. Fixes: d8cba25d2c68 ("crypto: qat - Intel(R) QAT driver framework") Cc: <stable@vger.kernel.org> Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-17crypto: qat - change SLAs cleanup flow at shutdownDamian Muszynski1-1/+19
The implementation of the Rate Limiting (RL) feature includes the cleanup of all SLAs during device shutdown. For each SLA, the firmware is notified of the removal through an admin message, the data structures that take into account the budgets are updated and the memory is freed. However, this explicit cleanup is not necessary as (1) the device is reset, and the firmware state is lost and (2) all RL data structures are freed anyway. In addition, if the device is unresponsive, for example after a PCI AER error is detected, the admin interface might not be available. This might slow down the shutdown sequence and cause a timeout in the recovery flows which in turn makes the driver believe that the device is not recoverable. Fix by replacing the explicit SLAs removal with just a free of the SLA data structures. Fixes: d9fb8408376e ("crypto: qat - add rate limiting feature to qat_4xxx") Cc: <stable@vger.kernel.org> Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-09crypto: qat - improve aer error reset handlingMun Chun Yep1-1/+25
Rework the AER reset and recovery flow to take into account root port integrated devices that gets reset between the error detected and the slot reset callbacks. In adf_error_detected() the devices is gracefully shut down. The worker threads are disabled, the error conditions are notified to listeners and through PFVF comms and finally the device is reset as part of adf_dev_down(). In adf_slot_reset(), the device is brought up again. If SRIOV VFs were enabled before reset, these are re-enabled and VFs are notified of restarting through PFVF comms. Signed-off-by: Mun Chun Yep <mun.chun.yep@intel.com> Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Reviewed-by: Markas Rapoportas <markas.rapoportas@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-09crypto: qat - limit heartbeat notificationsFurong Zhou2-3/+17
When the driver detects an heartbeat failure, it starts the recovery flow. Set a limit so that the number of events is limited in case the heartbeat status is read too frequently. Signed-off-by: Furong Zhou <furong.zhou@intel.com> Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Reviewed-by: Markas Rapoportas <markas.rapoportas@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Mun Chun Yep <mun.chun.yep@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-09crypto: qat - add auto reset on errorDamian Muszynski4-1/+49
Expose the `auto_reset` sysfs attribute to configure the driver to reset the device when a fatal error is detected. When auto reset is enabled, the driver resets the device when it detects either an heartbeat failure or a fatal error through an interrupt. This patch is based on earlier work done by Shashank Gupta. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Reviewed-by: Markas Rapoportas <markas.rapoportas@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Mun Chun Yep <mun.chun.yep@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-09crypto: qat - add fatal error notificationMun Chun Yep2-1/+9
Notify a fatal error condition and optionally reset the device in the following cases: * if the device reports an uncorrectable fatal error through an interrupt * if the heartbeat feature detects that the device is not responding This patch is based on earlier work done by Shashank Gupta. Signed-off-by: Mun Chun Yep <mun.chun.yep@intel.com> Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Reviewed-by: Markas Rapoportas <markas.rapoportas@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
generated by cgit v1.2.3 (git 2.39.0) at 2024-11-26 11:42:52 +0300