BMC/Intel-BMC/linux.git/drivers/virtio/virtio_ring.c, branch dev-4.6

virtio: Silence uninitialized variable warning

2016-05-01T12:50:08+00:00

Smatch complains that we might not initialize "queue". The issue is callers like setup_vq() from virtio_pci_modern.c where "num" could be something like 2 and "vring_align" is 64. In that case, vring_size() is less than PAGE_SIZE. It won't happen in real life, but we're getting the value of "num" from a register so it's not really possible to tell what value it holds with static analysis. Let's just silence the warning. Signed-off-by: Dan Carpenter Signed-off-by: Michael S. Tsirkin

vring: Use the DMA API on Xen

2016-03-02T15:01:59+00:00

Signed-off-by: Andy Lutomirski Signed-off-by: Michael S. Tsirkin Reviewed-by: David Vrabel Reviewed-by: Wei Liu

virtio: Add improved queue allocation API

2016-03-02T15:01:57+00:00

This leaves vring_new_virtqueue alone for compatbility, but it adds two new improved APIs: vring_create_virtqueue: Creates a virtqueue backed by automatically allocated coherent memory. (Some day it this could be extended to support non-coherent memory, too, if there ends up being a platform on which it's worthwhile.) __vring_new_virtqueue: Creates a virtqueue with a manually-specified layout. This should allow mic_virtio to work much more cleanly. Signed-off-by: Andy Lutomirski Signed-off-by: Michael S. Tsirkin

virtio_ring: Support DMA APIs

2016-03-02T15:01:57+00:00

virtio_ring currently sends the device (usually a hypervisor) physical addresses of its I/O buffers. This is okay when DMA addresses and physical addresses are the same thing, but this isn't always the case. For example, this never works on Xen guests, and it is likely to fail if a physical "virtio" device ever ends up behind an IOMMU or swiotlb. The immediate use case for me is to enable virtio on Xen guests. For that to work, we need vring to support DMA address translation as well as a corresponding change to virtio_pci or to another driver. Signed-off-by: Andy Lutomirski Signed-off-by: Michael S. Tsirkin

vring: Introduce vring_use_dma_api()

2016-03-02T15:01:56+00:00

This is a kludge, but no one has come up with a a better idea yet. We'll introduce DMA API support guarded by vring_use_dma_api(). Eventually we may be able to return true on more and more systems, and hopefully we can get rid of vring_use_dma_api() entirely some day. Signed-off-by: Andy Lutomirski Signed-off-by: Michael S. Tsirkin

virtio_ring: use virt_store_mb

2016-01-12T18:47:02+00:00

We need a full barrier after writing out event index, using virt_store_mb there seems better than open-coding. As usual, we need a wrapper to account for strong barriers. It's tempting to use this in vhost as well, for that, we'll need a variant of smp_store_mb that works on __user pointers. Signed-off-by: Michael S. Tsirkin Acked-by: Peter Zijlstra (Intel)

virtio_ring: shadow available ring flags & index

2015-12-07T15:28:11+00:00

Improves cacheline transfer flow of available ring header. Virtqueues are implemented as a pair of rings, one producer->consumer avail ring and one consumer->producer used ring; preceding the avail ring in memory are two contiguous u16 fields -- avail->flags and avail->idx. A producer posts work by writing to avail->idx and a consumer reads avail->idx. The flags and idx fields only need to be written by a producer CPU and only read by a consumer CPU; when the producer and consumer are running on different CPUs and the virtio_ring code is structured to only have source writes/sink reads, we can continuously transfer the avail header cacheline between 'M' states between cores. This flow optimizes core -> core bandwidth on certain CPUs. (see: "Software Optimization Guide for AMD Family 15h Processors", Section 11.6; similar language appears in the 10h guide and should apply to CPUs w/ exclusive caches, using LLC as a transfer cache) Unfortunately the existing virtio_ring code issued reads to the avail->idx and read-modify-writes to avail->flags on the producer. This change shadows the flags and index fields in producer memory; the vring code now reads from the shadows and only ever writes to avail->flags and avail->idx, allowing the cacheline to transfer core -> core optimally. In a concurrent version of vring_bench, the time required for 10,000,000 buffer checkout/returns was reduced by ~2% (average across many runs) on an AMD Piledriver (15h) CPU: (w/o shadowing): Performance counter stats for './vring_bench': 5,451,082,016 L1-dcache-loads ... 2.221477739 seconds time elapsed (w/ shadowing): Performance counter stats for './vring_bench': 5,405,701,361 L1-dcache-loads ... 2.168405376 seconds time elapsed The further away (in a NUMA sense) virtio producers and consumers are from each other, the more we expect to benefit. Physical implementations of virtio devices and implementations of virtio where the consumer polls vring avail indexes (vhost) should also benefit. Signed-off-by: Venkatesh Srinivas Signed-off-by: Michael S. Tsirkin

virtio: Do not drop __GFP_HIGH in alloc_indirect

2015-12-07T15:28:11+00:00

b92b1b89a33c ("virtio: force vring descriptors to be allocated from lowmem") tried to exclude highmem pages for descriptors so it cleared __GFP_HIGHMEM from a given gfp mask. The patch also cleared __GFP_HIGH which doesn't make much sense for this fix because __GFP_HIGH only controls access to memory reserves and it doesn't have any influence on the zone selection. Some of the call paths use GFP_ATOMIC and dropping __GFP_HIGH will reduce their changes for success because the lack of access to memory reserves. Signed-off-by: Michal Hocko Signed-off-by: Michael S. Tsirkin Acked-by: Will Deacon Reviewed-by: Mel Gorman

virtio: Avoid possible kernel panic if DEBUG is enabled.

2015-02-11T04:33:14+00:00

The virtqueue_add() calls START_USE() upon entry. The virtqueue_kick() is called if vq->num_added == (1 << 16) - 1 before calling END_USE(). The virtqueue_kick_prepare() called via virtqueue_kick() calls START_USE() upon entry, and will call panic() if DEBUG is enabled. Move this virtqueue_kick() call to after END_USE() call. Signed-off-by: Tetsuo Handa Signed-off-by: Rusty Russell

virtio_ring: coding style fix

2015-01-21T05:58:57+00:00

Most of our code has struct foo { } Fix one instances where ring is inconsistent. Signed-off-by: Michael S. Tsirkin Signed-off-by: Rusty Russell