kernel/linux.git/block/blk.h, branch v6.6.141

block: relax pgmap check in bio_add_page for compatible zone device pages

2026-05-17T15:13:49+00:00

[ Upstream commit 41c665aae2b5dbecddddcc8ace344caf630cc7a4 ] bio_add_page() and bio_integrity_add_page() reject pages from different dev_pagemaps entirely, returning 0 even when those pages have compatible DMA mapping requirements. This forces callers to start a new bio when buffers span pgmap boundaries, even though the pages could safely coexist as separate bvec entries. This matters for guests where memory is registered through devm_memremap_pages() with MEMORY_DEVICE_GENERIC in multiple calls, creating separate dev_pagemaps for each chunk. When a direct I/O buffer spans two such chunks, bio_add_page() rejects the second page, forcing an unnecessary bio split or I/O failure. Introduce zone_device_pages_compatible() in blk.h to check whether two pages can coexist in the same bio as separate bvec entries. The block DMA iterator (blk_dma_map_iter_start) caches the P2PDMA mapping state from the first segment and applies it to all others, so P2PDMA pages from different pgmaps must not be mixed, and neither must P2PDMA and non-P2PDMA pages. All other combinations (MEMORY_DEVICE_GENERIC pages from different pgmaps, or MEMORY_DEVICE_GENERIC with normal RAM) use the same dma_map_phys path and are safe. Replace the blanket zone_device_pages_have_same_pgmap() rejection with zone_device_pages_compatible(), while keeping zone_device_pages_have_same_pgmap() as a merge guard. Pages from different pgmaps can be added as separate bvec entries but must not be coalesced into the same segment, as that would make it impossible to recover the correct pgmap via page_pgmap(). Fixes: 49580e690755 ("block: add check when merging zone device pages") Cc: stable@vger.kernel.org Signed-off-by: Naman Jain Reviewed-by: Christoph Hellwig Link: https://patch.msgid.link/20260410153414.4159050-3-namjain@linux.microsoft.com Signed-off-by: Jens Axboe [ restructured combined `if` into explicit `bv` block ] Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

block: add pgmap check to biovec_phys_mergeable

2026-05-17T15:13:39+00:00

commit 13920e4b7b784b40cf4519ff1f0f3e513476a499 upstream. biovec_phys_mergeable() is used by the request merge, DMA mapping, and integrity merge paths to decide if two physically contiguous bvec segments can be coalesced into one. It currently has no check for whether the segments belong to different dev_pagemaps. When zone device memory is registered in multiple chunks, each chunk gets its own dev_pagemap. A single bio can legitimately contain bvecs from different pgmaps -- iov_iter_extract_bvecs() breaks at pgmap boundaries but the outer loop in bio_iov_iter_get_pages() continues filling the same bio. If such bvecs are physically contiguous, biovec_phys_mergeable() will coalesce them, making it impossible to recover the correct pgmap for the merged segment via page_pgmap(). Add a zone_device_pages_have_same_pgmap() check to prevent merging bvec segments that span different pgmaps. Fixes: 49580e690755 ("block: add check when merging zone device pages") Cc: stable@vger.kernel.org Reviewed-by: Christoph Hellwig Signed-off-by: Naman Jain Link: https://patch.msgid.link/20260410153414.4159050-2-namjain@linux.microsoft.com Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

block: support to account io_ticks precisely

2024-06-12T09:11:35+00:00

[ Upstream commit 99dc422335d8b2bd4d105797241d3e715bae90e9 ] Currently, io_ticks is accounted based on sampling, specifically update_io_ticks() will always account io_ticks by 1 jiffies from bdev_start_io_acct()/blk_account_io_start(), and the result can be inaccurate, for example(HZ is 250): Test script: fio -filename=/dev/sda -bs=4k -rw=write -direct=1 -name=test -thinktime=4ms Test result: util is about 90%, while the disk is really idle. This behaviour is introduced by commit 5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting"), however, there was a key point that is missed that this patch also improve performance a lot: Before the commit: part_round_stats: if (part->stamp != now) stats |= 1; part_in_flight() -> there can be lots of task here in 1 jiffies. part_round_stats_single() __part_stat_add() part->stamp = now; After the commit: update_io_ticks: stamp = part->bd_stamp; if (time_after(now, stamp)) if (try_cmpxchg()) __part_stat_add() -> only one task can reach here in 1 jiffies. Hence in order to account io_ticks precisely, we only need to know if there are IO inflight at most once in one jiffies. Noted that for rq-based device, iterating tags should not be used here because 'tags->lock' is grabbed in blk_mq_find_and_get_req(), hence part_stat_lock_inc/dec() and part_in_flight() is used to trace inflight. The additional overhead is quite little: - per cpu add/dec for each IO for rq-based device; - per cpu sum for each jiffies; And it's verified by null-blk that there are no performance degration under heavy IO pressure. Fixes: 5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting") Signed-off-by: Yu Kuai Link: https://lore.kernel.org/r/20240509123717.3223892-2-yukuai1@huaweicloud.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin

block: Add some exports for bcachefs

2023-08-14T21:40:42+00:00

- bio_set_pages_dirty(), bio_check_pages_dirty() - dio path - blk_status_to_str() - error messages - bio_add_folio() - this should definitely be exported for everyone, it's the modern version of bio_add_page() Signed-off-by: Kent Overstreet Cc: linux-block@vger.kernel.org Cc: Jens Axboe Signed-off-by: Kent Overstreet Link: https://lore.kernel.org/r/20230813182636.2966159-2-kent.overstreet@linux.dev Signed-off-by: Jens Axboe

block: make bvec_try_merge_hw_page() non-static

2023-08-09T22:05:35+00:00

This will be used for multi-page configuration for integrity payload. Cc: Christoph Hellwig Cc: Martin K. Petersen Reviewed-by: Christoph Hellwig Signed-off-by: Jinyoung Choi Tested-by: "Martin K. Petersen" Reviewed-by: "Martin K. Petersen" Link: https://lore.kernel.org/r/20230803024827epcms2p838d9e9131492c86a159fff25d195658f@epcms2p8 Signed-off-by: Jens Axboe

blk-flush: count inflight flush_data requests

2023-07-17T14:18:21+00:00

The flush state machine use a double list to link all inflight flush_data requests, to avoid issuing separate post-flushes for these flush_data requests which shared PREFLUSH. So we can't reuse rq->queuelist, this is why we need rq->flush.list In preparation of the next patch that reuse rq->queuelist for flush state machine, we change the double linked list to unsigned long counter, which count all inflight flush_data requests. This is ok since we only need to know if there is any inflight flush_data request, so unsigned long counter is good. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20230717040058.3993930-4-chengming.zhou@linux.dev Signed-off-by: Jens Axboe

block: remove BIO_PAGE_REFFED

2023-06-16T16:08:09+00:00

Now that all block direct I/O helpers use page pinning, this flag is unused. Signed-off-by: Christoph Hellwig Reviewed-by: Christian Brauner Reviewed-by: Johannes Thumshirn Reviewed-by: David Howells Link: https://lore.kernel.org/r/20230614140341.521331-4-hch@lst.de Signed-off-by: Jens Axboe

block: replace fmode_t with a block-specific type for block open flags

2023-06-12T14:04:05+00:00

The only overlap between the block open flags mapped into the fmode_t and other uses of fmode_t are FMODE_READ and FMODE_WRITE. Define a new blk_mode_t instead for use in blkdev_get_by_{dev,path}, ->open and ->ioctl and stop abusing fmode_t. Signed-off-by: Christoph Hellwig Acked-by: Jack Wang [rnbd] Reviewed-by: Hannes Reinecke Reviewed-by: Christian Brauner Link: https://lore.kernel.org/r/20230608110258.189493-28-hch@lst.de Signed-off-by: Jens Axboe

block: remove unused fmode_t arguments from ioctl handlers

2023-06-12T14:04:04+00:00

A few ioctl handlers have fmode_t arguments that are entirely unused, remove them. Signed-off-by: Christoph Hellwig Acked-by: Christian Brauner Reviewed-by: Hannes Reinecke Link: https://lore.kernel.org/r/20230608110258.189493-27-hch@lst.de Signed-off-by: Jens Axboe

block: move a few internal definitions out of blkdev.h

2023-06-12T14:04:04+00:00

All these helpers are only used in core block code, so move them out of the public header. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Acked-by: Christian Brauner Link: https://lore.kernel.org/r/20230608110258.189493-26-hch@lst.de Signed-off-by: Jens Axboe