10G NIC performance: VFIO vs virtio - Revision history

Bsd: Add networking category

2015-05-16T20:52:45Z

Add networking category

← Older revision		Revision as of 16:52, 16 May 2015
Line 66:		Line 66:
	<br />If not use <nowiki>'-device virtio-net-pci'</nowiki> option, performance will be 3.6 Gbps.		<br />If not use <nowiki>'-device virtio-net-pci'</nowiki> option, performance will be 3.6 Gbps.

	[[Category:VFIO]][[Category:Results]][[Category:Virtio]]		[[Category:VFIO]][[Category:Results]][[Category:Virtio]][[Category:Networking]]

Bsd: Add categories

2015-05-16T20:25:06Z

Add categories

← Older revision		Revision as of 16:25, 16 May 2015
Line 65:		Line 65:
	<br />'''<nowiki>qemu-kvm -m 16G -smp 8 -net nic,model=virtio -net tap,script=/etc/qemu-ifup -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio –nographic</nowiki>'''		<br />'''<nowiki>qemu-kvm -m 16G -smp 8 -net nic,model=virtio -net tap,script=/etc/qemu-ifup -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio –nographic</nowiki>'''
	<br />If not use <nowiki>'-device virtio-net-pci'</nowiki> option, performance will be 3.6 Gbps.		<br />If not use <nowiki>'-device virtio-net-pci'</nowiki> option, performance will be 3.6 Gbps.

			[[Category:VFIO]][[Category:Results]][[Category:Virtio]]

Robert: Robert moved page VFIO vs virtio to 10G NIC performance: VFIO vs virtio: More accurate name

2015-05-12T02:54:39Z

Robert moved page VFIO vs virtio to 10G NIC performance: VFIO vs virtio: More accurate name

← Older revision

Revision as of 22:54, 11 May 2015

Robert at 02:53, 12 May 2015

2015-05-12T02:53:15Z

← Older revision		Revision as of 22:53, 11 May 2015
Line 4:		Line 4:

	2 Intel Grantley-EP platforms (Xeon E5-2697 v3) connected by 10G link; memory 96 G.<br />		2 Intel Grantley-EP platforms (Xeon E5-2697 v3) connected by 10G link; memory 96 G.<br />
	NIC: Intel 82599ES<br />		NIC: Intel 82599ES [http://ark.intel.com/products/41282/Intel-82599ES-10-Gigabit-Ethernet-Controller]<br />
	Test Tool: iperf <br />		Test Tool: iperf <br />
	OS: RHEL 7.1 <br />		OS: RHEL 7.1 <br />
Line 13:		Line 13:
	*With virtio approach, if proper configured (details see below), network performance can also achieve '''9.4''' Gbps; otherwise, poor performance will be '''3.6''' Gbps.		*With virtio approach, if proper configured (details see below), network performance can also achieve '''9.4''' Gbps; otherwise, poor performance will be '''3.6''' Gbps.

			=== Some references first ===
			SR-IOV[http://www.intel.com/content/www/us/en/network-adapters/virtualization.html]<br />
			vt-d assignment[[How_to_assign_devices_with_VT-d_in_KVM]]
	----		----

Robert: finalize

2015-05-11T11:00:16Z

finalize

← Older revision		Revision as of 07:00, 11 May 2015
Line 9:		Line 9:

	== Result summary ==		== Result summary ==
	*In native environment, iperf can get 9.4 Gbps throughput. Since iperf is a SW packet generator and normal process shall it be, this a reasonable number.		*In native environment, iperf can get '''9.4''' Gbps throughput. Since iperf is a SW packet generator and normal process shall it be, this a reasonable number.
	*With VFIO passthrough, network performance is also 9.4 Gbps; i.e., we cannot observe overhead in virtualization environment with VFIO passthrough method, in context of typical SW network user application.		*With VFIO passthrough, network performance is also '''9.4''' Gbps; i.e., we cannot observe overhead in virtualization environment with VFIO passthrough method, in context of typical SW network user application.
	*With virtio approach, if proper configured (details see below), network performance can also achieve 9.4 Gbps.		*With virtio approach, if proper configured (details see below), network performance can also achieve '''9.4''' Gbps; otherwise, poor performance will be '''3.6''' Gbps.

	----		----
Line 24:		Line 24:

	=== Check if your NIC supports SR-IOV ===		=== Check if your NIC supports SR-IOV ===
	lspci -s <NIC_BDF> -vvv \| grep -i "Single Root I/O Virtualization"		'''<nowiki>lspci -s <NIC_BDF> -vvv \| grep -i "Single Root I/O Virtualization"</nowiki>'''

	=== Assign the VF to a guest ===		=== Assign the VF to a guest ===

Robert: More fine tune

2015-05-11T10:56:03Z

More fine tune

← Older revision		Revision as of 06:56, 11 May 2015
Line 3:		Line 3:
	== Test Topology ==		== Test Topology ==

	2 Intel Grantley-EP platforms (Xeon E5-2697 v3) connected by 10G link; memory 96 G.		2 Intel Grantley-EP platforms (Xeon E5-2697 v3) connected by 10G link; memory 96 G.<br />
			NIC: Intel 82599ES<br />
	NIC: Intel 82599ES		Test Tool: iperf <br />
			OS: RHEL 7.1 <br />
	Test Tool: iperf

	OS: RHEL 7.1

	== Result summary ==		== Result summary ==
Line 61:		Line 58:
	'''<nowiki>qemu-kvm -m 16G -smp 8 -device virtio-net-pci,netdev=net0 -netdev tap,id=net0,script=/etc/qemu-ifup -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio –nographic</nowiki>'''		'''<nowiki>qemu-kvm -m 16G -smp 8 -device virtio-net-pci,netdev=net0 -netdev tap,id=net0,script=/etc/qemu-ifup -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio –nographic</nowiki>'''

	==== Poor performance configuration ====<br />		<br />
	'''<nowiki>qemu-kvm -m 16G -smp 8 -net nic,model=virtio -net tap,script=/etc/qemu-ifup -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio –nographic</nowiki>'''		==== Poor performance configuration ====
	If not use <nowiki>'-device virtio-net-pci'</nowiki> option, performance will be 3.6 Gbps.		<br />'''<nowiki>qemu-kvm -m 16G -smp 8 -net nic,model=virtio -net tap,script=/etc/qemu-ifup -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio –nographic</nowiki>'''
			<br />If not use <nowiki>'-device virtio-net-pci'</nowiki> option, performance will be 3.6 Gbps.

Robert: virtio part complete

2015-05-11T10:50:22Z

virtio part complete

@@ Line 15: / Line 15: @@
 *With VFIO passthrough, network performance is also 9.4 Gbps; i.e., we cannot observe overhead in virtualization environment with VFIO passthrough method, in context of typical SW network user application.
 *With virtio approach, if proper configured (details see below), network performance can also achieve 9.4 Gbps.
 Here is the details about each kind of configuration.
@@ Line 45: / Line 47: @@
 <br />'-net none' tells qemu not emulate network devices
 <br />'-device vfio-pci,host=' designate a vfio-pci device and the device's host BDF

Robert: VFIO method completes

2015-05-11T10:44:56Z

VFIO method completes

← Older revision		Revision as of 06:44, 11 May 2015
Line 40:		Line 40:
	Now you can see this device is bound to vfio-pci driver<br />		Now you can see this device is bound to vfio-pci driver<br />
	'''lspci -s <vf_BDF> -k'''		'''lspci -s <vf_BDF> -k'''

			Create guest with direct passthrough via VFIO framework<br />
			'''<nowiki>qemu-kvm -m 16G -smp 8 -net none -device vfio-pci,host=81:10.0 -drive file=/var/lib/libvirt/images/rhel7.1.img,if=virtio -nographic</nowiki>'''
			<br />'-net none' tells qemu not emulate network devices
			<br />'-device vfio-pci,host=' designate a vfio-pci device and the device's host BDF

Robert at 10:00, 11 May 2015

2015-05-11T10:00:02Z

← Older revision		Revision as of 06:00, 11 May 2015
Line 30:		Line 30:
	Unbind from igbvf driver and Bind to VFIO driver		Unbind from igbvf driver and Bind to VFIO driver
	#unbind from previous driver (take igbvf device for example)		#unbind from previous driver (take igbvf device for example)
	echo <vf_BDF> > /sys/bus/pci/device/<vf_BDF>/driver/unbind		#;echo <vf_BDF> > /sys/bus/pci/device/<vf_BDF>/driver/unbind
	lspci -s <vf_BDF> -n //to get its number		#;lspci -s <vf_BDF> -n //to get its number
	//it will return like below		#://it will return like below
	0a:13.3 0200: 8086:1520 (rev 01)		#;<nowiki>0a:13.3 0200: 8086:1520 (rev 01)</nowiki>
	//8086 1520 is its numeric number		#://8086 1520 is its numeric number

	#bind to vfio-pci driver		#bind to vfio-pci driver
	echo 8086 1520 > /sys/bus/pci/drivers/vfio-pci/new_id		#;echo 8086 1520 > /sys/bus/pci/drivers/vfio-pci/new_id

	Now you can see this device is bound to vfio-pci driver		Now you can see this device is bound to vfio-pci driver<br />
	lspci -s <vf_BDF> -k		'''lspci -s <vf_BDF> -k'''

Robert: fine tune

2015-05-11T09:52:44Z

fine tune

← Older revision		Revision as of 05:52, 11 May 2015
Line 32:		Line 32:
	echo <vf_BDF> > /sys/bus/pci/device/<vf_BDF>/driver/unbind		echo <vf_BDF> > /sys/bus/pci/device/<vf_BDF>/driver/unbind
	lspci -s <vf_BDF> -n //to get its number		lspci -s <vf_BDF> -n //to get its number
	//it will return like below		//it will return like below
	0a:13.3 0200: 8086:1520 (rev 01)		0a:13.3 0200: 8086:1520 (rev 01)
	//8086 1520 is its numeric number		//8086 1520 is its numeric number

	#bind to vfio-pci driver		#bind to vfio-pci driver