From KVM
m (public bridge)
(Private Virtual Bridge: manually created bridge must be (may need to be?) brought up to be useful)
 
(19 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= Setting guest network =
+
= Configuring Guest Networking =
  
Guest (VM) networking in kvm is the same as in qemu, so it is possible to refer to other documentations about networking for qemu. This page will try to explain how to configure the most frequent types of network needed.
+
Guest (VM) networking in kvm is the same as in qemu, so it is possible to refer to other documentation about networking in qemu. This page will try to explain how to configure the most frequent types of networking needed.
  
  
 
== User Networking ==
 
== User Networking ==
 +
  
 
'''Use case:'''
 
'''Use case:'''
Line 11: Line 12:
 
* You are ready to take a huge performance hit.
 
* You are ready to take a huge performance hit.
 
* Warning: User networking does not support a number of networking features like ICMP.  Certain applications (like ping) may not function properly.
 
* Warning: User networking does not support a number of networking features like ICMP.  Certain applications (like ping) may not function properly.
 +
  
 
'''Prerequisites:'''
 
'''Prerequisites:'''
 
* You need kvm up and running
 
* You need kvm up and running
* If you don't want to run as root, the user you want to use needs to have rw access to /dev/kvm
+
* If you don't want to run as root, then the user needs to have rw access to /dev/kvm
* If you want to be able to access the internet or a local network, your host system must be able to access the internet or the local network
+
* In order for the guest to be able to access the internet or a local network, the host system must be able to access these resources as well
 +
 
  
 
'''Solution:'''
 
'''Solution:'''
* simply run your guest without specifying network parameters, which by default will create user-lever (a.k.a slirp) networking:
+
* Simply run your guest without specifying network parameters, which by default will create user-level (a.k.a slirp) networking:
 
  qemu-system-x86_64 -hda /path/to/hda.img
 
  qemu-system-x86_64 -hda /path/to/hda.img
 +
  
 
'''Notes:'''
 
'''Notes:'''
Line 26: Line 30:
 
* The default is equivalent to this explicit setup:
 
* The default is equivalent to this explicit setup:
 
  qemu-system-x86_64 -hda /path/to/hda.img -netdev user,id=user.0 -device e1000,netdev=user.0
 
  qemu-system-x86_64 -hda /path/to/hda.img -netdev user,id=user.0 -device e1000,netdev=user.0
* user.0 identifier above is just to connect the two halves into one, you may use any identifier you wish, such as "n" or "net0".
+
* The user.0 identifier above is just to connect the two halves into one. You may use any identifier you wish, such as "n" or "net0".
* Use rtl8139 instead of e1000 to get 8139-series NIC.
+
* Use rtl8139 instead of e1000 to get an rtl8139-based network interface.
* You can still access one specific port on the guest using the "hostfwd" option. This means e.g. if you want to transport a file with scp from host to guest, start the guest with "-device e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22". Now you are forwarding the host port 5555 to the guest port 22. After starting up the guest, you can transport a file with e.g. "scp -P 5555 file.txt root@localhost:/tmp" from host to guest. Or you can also use other address of the host to connect to.
+
* You can still access one specific port on the guest using the "hostfwd" option. This means e.g. if you want to transport a file with scp from host to guest, start the guest with "-device e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22". Now you are forwarding the host port 5555 to the guest port 22. After starting up the guest, you can transport a file with e.g. "scp -P 5555 file.txt root@localhost:/tmp" from host to guest. Or you can also use the other address of the host to connect to.
 
+
  
== private virtual bridge ==
+
== Private Virtual Bridge ==
  
 
'''Use case:'''
 
'''Use case:'''
Line 38: Line 41:
 
'''Prerequisites:'''
 
'''Prerequisites:'''
 
* You need kvm up and running
 
* You need kvm up and running
* If you don't want to run as root, the user you want to use needs to have rw access to /dev/kvm
+
* If you don't want to run as root, then the user needs to have rw access to /dev/kvm
* You need the following commands installed on your system, and if you don't want to run as root, the user you want to use needs to be able to sudo the following command:
+
* The following commands must be installed on the host system and executed as root:
  /sbin/ip
+
  ip
  /usr/sbin/brctl
+
  brctl (deprecated). Use ip link instead
  /usr/sbin/tunctl
+
  tunctl (deprecated). Use ip tuntap and ip link instead
  
 
'''Solution:'''
 
'''Solution:'''
  
 
* You need to create a bridge, e-g:
 
* You need to create a bridge, e-g:
  sudo /usr/sbin/brctl addbr br0
+
  # ip link add br0 type bridge ; ifconfig br0 up
 +
# brctl addbr br0 (deprecated)
  
* You need a qemu-ifup script containing the following:
+
* You need a qemu-ifup script containing the following (run as root):
 
  #!/bin/sh
 
  #!/bin/sh
 
  set -x
 
  set -x
Line 56: Line 60:
 
   
 
   
 
  if [ -n "$1" ];then
 
  if [ -n "$1" ];then
         /usr/bin/sudo /usr/sbin/tunctl -u `whoami` -t $1
+
         # tunctl -u `whoami` -t $1 (use ip tuntap instead!)
         /usr/bin/sudo /sbin/ip link set $1 up
+
        ip tuntap add $1 mode tap user `whoami`
 +
         ip link set $1 up
 
         sleep 0.5s
 
         sleep 0.5s
         /usr/bin/sudo /usr/sbin/brctl addif $switch $1
+
         # brctl addif $switch $1 (use ip link instead!)
 +
        ip link set $1 master $switch
 
         exit 0
 
         exit 0
 
  else
 
  else
Line 75: Line 81:
  
 
'''Notes:'''
 
'''Notes:'''
* If you don't want to run as root, the qemu-ifup must be executable by the user you want to use
+
* If you don't want to run qemu-ifup as root, then consider using sudo
 
* You can either create a system-wide qemu-ifup in /etc/qemu-ifup or use another one. In the latter case, run
 
* You can either create a system-wide qemu-ifup in /etc/qemu-ifup or use another one. In the latter case, run
 
  qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0,script=/path/to/qemu-ifup
 
  qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0,script=/path/to/qemu-ifup
Line 81: Line 87:
 
* Each guest on the private virtual network must have a different MAC address
 
* Each guest on the private virtual network must have a different MAC address
  
 +
== Public Bridge ==
  
== public bridge ==
+
'''WARNING:''' The method shown here will not work with all wireless drivers as they might not support bridging.
 
+
'''WARNING:''' The method shown here will not work with most (if not all) wireless drivers as these do not support bridging.
+
  
 
'''Use case:'''
 
'''Use case:'''
  
* You want to assign an IP address to your virtual machines and make them accessible from your local network
+
* You want to assign IP addresses to your virtual machines and make them accessible from your local network
* You also want performance out of your virtual machine.
+
* You also want performance out of your virtual machine
 +
 
  
 
'''Prerequisites:'''
 
'''Prerequisites:'''
Line 95: Line 101:
 
* If you don't want to run kvm as root, then the user must have rw access to /dev/kvm
 
* If you don't want to run kvm as root, then the user must have rw access to /dev/kvm
 
* The following commands must be installed on the host system and executed as root:
 
* The following commands must be installed on the host system and executed as root:
  /sbin/ip
+
  ip
  /usr/sbin/brctl
+
  brctl (deprecated, use ip link instead)
  /usr/sbin/tunctl
+
  tunctl (deprecated, use ip tuntap instead)
  
 
* Your host system must be able to access the internet or the local network
 
* Your host system must be able to access the internet or the local network
 +
  
 
'''Solution 1: Using Distribution-Specific Scripts'''
 
'''Solution 1: Using Distribution-Specific Scripts'''
 
{|border="1"
 
{|border="1"
!RedHat's way
+
!RedHat
!Debian's way
+
!Debian
!SuSE's way
+
!SuSE
 
|-
 
|-
 
|
 
|
Line 139: Line 146:
  
 
* /etc/init.d/networking restart
 
* /etc/init.d/networking restart
* The bridge br0 should get the ip address (either static/dhcp) while the physical eth0 is left without an ip address.
+
* The bridge br0 should get the IP address (either static/dhcp) while the physical eth0 is left without an IP address.
 +
 
  
 
'''VLANs'''
 
'''VLANs'''
Line 156: Line 164:
  
 
* You need to create a bridge, e-g:
 
* You need to create a bridge, e-g:
  # /usr/sbin/brctl addbr br0
+
  # ip link add br0 type bridge
 +
# brctl addbr br0 (deprecated, use ip link instead!)
  
 
* Add one of your physical interface to the bridge, e-g for eth0:
 
* Add one of your physical interface to the bridge, e-g for eth0:
  # /usr/sbin/brctl addif br0 eth0
+
  # ip link set eth0 master br0
 +
# brctl addif br0 eth0 (deprecated, use ip link instead!)
  
 
* You need a qemu-ifup script containing the following (run as root):
 
* You need a qemu-ifup script containing the following (run as root):
Line 169: Line 179:
 
   
 
   
 
  if [ -n "$1" ];then
 
  if [ -n "$1" ];then
         /usr/sbin/tunctl -u `whoami` -t $1
+
         #tunctl -u `whoami` -t $1
         /sbin/ip link set $1 up
+
         ip tuntap add $1 mode tap user `whoami`
 +
        ip link set $1 up
 
         sleep 0.5s
 
         sleep 0.5s
         /usr/sbin/brctl addif $switch $1
+
         #brctl addif $switch $1
 +
        ip link set $1 master $switch
 
         exit 0
 
         exit 0
 
  else
 
  else
Line 187: Line 199:
 
* Run each guest with the following, replacing $macaddress with the value from the previous step
 
* Run each guest with the following, replacing $macaddress with the value from the previous step
 
  qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0
 
  qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0
 +
  
 
'''Notes:'''
 
'''Notes:'''
* If you don't want to run as root, the qemu-ifup must be executable by the user you want to use
+
* If you don't want to run qemu-ifup as root, then consider using sudo
 +
* Each guest on the network must have a different MAC address
 
* You can either create a system-wide qemu-ifup in /etc/qemu-ifup or use another one. In the latter case, run
 
* You can either create a system-wide qemu-ifup in /etc/qemu-ifup or use another one. In the latter case, run
 
  qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0,script=/path/to/qemu-ifup
 
  qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0,script=/path/to/qemu-ifup
  
* Each guest on the network must have a different MAC address
+
== Routing with iptables ==
  
== iptables/routing ==
+
With this method, you can connect your guest vm to a tap device in your host. Then you can set iptables rules in your host so that it acts as a router and firewall for your guest.
  
you can also connect your guest vm to a tap in your host. then setting iptables rules in your host to become a router + firewall for your vm.
+
Routing is done simply by setting the default route on the client to the IP address of the host, allowing IP forwarding, and setting a route to the tap device of the client on the host.
  
Routing would be done simply by creating the default route on the client to the IP of the host (and allowing IP forwarding) and setting a route to the tap? device of the client on the host.
 
  
Test the setup beforehand:
+
*Host-side: Allow IPv4 forwarding and add a route to the guest (could be put in a script, but the route has to be added after the guest has started):
 
+
*Hostside: Allow IPv4 forwarding and add route to client (could be put in a script - route has to be added after the client has started):
+
  
 
  sysctl -w net.ipv4.ip_forward=1                # allow forwarding of IPv4
 
  sysctl -w net.ipv4.ip_forward=1                # allow forwarding of IPv4
 
  route add -host <ip-of-client> dev <tap-device> # add route to the client
 
  route add -host <ip-of-client> dev <tap-device> # add route to the client
  
*Clientside: Default GW of the client is of course then the host (<ip-of-host> has to be in same subnet as <ip-of-client> ...):
+
 
 +
*Guest-side: Set the default gateway to the IP address of the host (make sure the host and guest IP addresses are in the same subnet):
  
 
  route add default gw <ip-of-host>
 
  route add default gw <ip-of-host>
  
*Clientside v2: If you host IP is not on the same subnet as <ip-of-client>, then you must manually add the route to host before you create default route:
+
 
 +
*Note: If the host is not on the same subnet as the guest, then you must manually add the route to the host before you create the default route:
  
 
  route add -host <ip-of-host> dev <network-interface>
 
  route add -host <ip-of-host> dev <network-interface>
 
  route add default gw <ip-of-host>
 
  route add default gw <ip-of-host>
  
 +
== VDE ==
  
== vde ==
+
Another option is using VDE (Virtual Distributed Ethernet).
 
+
Another option is using vde (virtual distributed ethernet).
+
  
 +
More information will be provided later.
  
== performance ==
+
== Performance ==
  
 
Data on benchmarking results should go in here.
 
Data on benchmarking results should go in here.
Line 229: Line 242:
 
[[Networking Performance]].
 
[[Networking Performance]].
  
Some 10G NIC performance comparison between VFIO passthrough and virtio [[VFIO vs virtio]]
+
Some 10G NIC performance comparisons between VFIO passthrough and virtio are discussed in [[VFIO vs virtio]].
  
 
== Compatibility ==
 
== Compatibility ==

Latest revision as of 03:11, 4 December 2018

Configuring Guest Networking

Guest (VM) networking in kvm is the same as in qemu, so it is possible to refer to other documentation about networking in qemu. This page will try to explain how to configure the most frequent types of networking needed.


User Networking

Use case:

  • You want a simple way for your virtual machine to access to the host, to the internet or to resources available on your local network.
  • You don't need to access your guest from the network or from another guest.
  • You are ready to take a huge performance hit.
  • Warning: User networking does not support a number of networking features like ICMP. Certain applications (like ping) may not function properly.


Prerequisites:

  • You need kvm up and running
  • If you don't want to run as root, then the user needs to have rw access to /dev/kvm
  • In order for the guest to be able to access the internet or a local network, the host system must be able to access these resources as well


Solution:

  • Simply run your guest without specifying network parameters, which by default will create user-level (a.k.a slirp) networking:
qemu-system-x86_64 -hda /path/to/hda.img


Notes:

  • The IP address can be automatically assigned to the guest thanks to the DHCP service integrated in QEMU
  • If you run multiple guests on the host, you don't need to specify a different MAC address for each guest
  • The default is equivalent to this explicit setup:
qemu-system-x86_64 -hda /path/to/hda.img -netdev user,id=user.0 -device e1000,netdev=user.0
  • The user.0 identifier above is just to connect the two halves into one. You may use any identifier you wish, such as "n" or "net0".
  • Use rtl8139 instead of e1000 to get an rtl8139-based network interface.
  • You can still access one specific port on the guest using the "hostfwd" option. This means e.g. if you want to transport a file with scp from host to guest, start the guest with "-device e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22". Now you are forwarding the host port 5555 to the guest port 22. After starting up the guest, you can transport a file with e.g. "scp -P 5555 file.txt root@localhost:/tmp" from host to guest. Or you can also use the other address of the host to connect to.

Private Virtual Bridge

Use case:

  • You want to set up a private network between 2 or more virtual machines. This network won't be seen from the other virtual machines nor from the real network.

Prerequisites:

  • You need kvm up and running
  • If you don't want to run as root, then the user needs to have rw access to /dev/kvm
  • The following commands must be installed on the host system and executed as root:
ip
brctl (deprecated). Use ip link instead
tunctl (deprecated). Use ip tuntap and ip link instead

Solution:

  • You need to create a bridge, e-g:
# ip link add br0 type bridge ; ifconfig br0 up
# brctl addbr br0 (deprecated)
  • You need a qemu-ifup script containing the following (run as root):
#!/bin/sh
set -x

switch=br0

if [ -n "$1" ];then
        # tunctl -u `whoami` -t $1 (use ip tuntap instead!)
        ip tuntap add $1 mode tap user `whoami`
        ip link set $1 up
        sleep 0.5s
        # brctl addif $switch $1 (use ip link instead!)
        ip link set $1 master $switch
        exit 0
else
        echo "Error: no interface specified"
        exit 1
fi
  • Generate a MAC address, either manually or using:
#!/bin/bash
# generate a random mac address for the qemu nic
printf 'DE:AD:BE:EF:%02X:%02X\n' $((RANDOM%256)) $((RANDOM%256))
  • Run each guest with the following, replacing $macaddress with the value from the previous step
qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0

Notes:

  • If you don't want to run qemu-ifup as root, then consider using sudo
  • You can either create a system-wide qemu-ifup in /etc/qemu-ifup or use another one. In the latter case, run
qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0,script=/path/to/qemu-ifup
  • Each guest on the private virtual network must have a different MAC address

Public Bridge

WARNING: The method shown here will not work with all wireless drivers as they might not support bridging.

Use case:

  • You want to assign IP addresses to your virtual machines and make them accessible from your local network
  • You also want performance out of your virtual machine


Prerequisites:

  • You need kvm up and running
  • If you don't want to run kvm as root, then the user must have rw access to /dev/kvm
  • The following commands must be installed on the host system and executed as root:
ip
brctl (deprecated, use ip link instead)
tunctl (deprecated, use ip tuntap instead)
  • Your host system must be able to access the internet or the local network


Solution 1: Using Distribution-Specific Scripts

RedHat Debian SuSE
  • Edit /etc/sysconfig/network-scripts/ifcfg-eth0
    • comment out BOOTPROTO
    • Add BRIDGE=br0
  • Create /etc/sysconfig/network-scripts/ifcfg-br0
    • The content should be:
DEVICE=br0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Bridge

/etc/network/interfaces

# Replace old eth0 config with br0
auto eth0 br0
# Use old eth0 config for br0, plus bridge stuff
iface br0 inet dhcp
    bridge_ports    eth0
    bridge_stp      off
    bridge_maxwait  0
    bridge_fd       0
  • Start YaST
  • Go to Network Configuration
  • Add new device -> Bridge
  • Tick your existing network device
  • done
  • /etc/init.d/networking restart
  • The bridge br0 should get the IP address (either static/dhcp) while the physical eth0 is left without an IP address.


VLANs

Please note that the rtl8139 virtual network interface driver does not support VLANs. If you want to use VLANs with your virtual machine, you must use another virtual network interface like virtio.

When using VLANs on a setup like this and no traffic is getting through to your guest(s), you might want to do:

# cd /proc/sys/net/bridge
# ls
bridge-nf-call-arptables  bridge-nf-call-iptables
bridge-nf-call-ip6tables  bridge-nf-filter-vlan-tagged
# for f in bridge-nf-*; do echo 0 > $f; done


Solution 2: Manual Configuration

  • You need to create a bridge, e-g:
# ip link add br0 type bridge
# brctl addbr br0 (deprecated, use ip link instead!)
  • Add one of your physical interface to the bridge, e-g for eth0:
# ip link set eth0 master br0
# brctl addif br0 eth0 (deprecated, use ip link instead!)
  • You need a qemu-ifup script containing the following (run as root):
#!/bin/sh
set -x

switch=br0

if [ -n "$1" ];then
        #tunctl -u `whoami` -t $1
        ip tuntap add $1 mode tap user `whoami`
        ip link set $1 up
        sleep 0.5s
        #brctl addif $switch $1
        ip link set $1 master $switch
        exit 0
else
        echo "Error: no interface specified"
        exit 1
fi
  • Generate a MAC address, either manually or using:
#!/bin/sh
# generate a random mac address for the qemu nic
printf 'DE:AD:BE:EF:%02X:%02X\n' $((RANDOM%256)) $((RANDOM%256))
  • Run each guest with the following, replacing $macaddress with the value from the previous step
qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0


Notes:

  • If you don't want to run qemu-ifup as root, then consider using sudo
  • Each guest on the network must have a different MAC address
  • You can either create a system-wide qemu-ifup in /etc/qemu-ifup or use another one. In the latter case, run
qemu-system-x86_64 -hda /path/to/hda.img -device e1000,netdev=net0,mac=$macaddress -netdev tap,id=net0,script=/path/to/qemu-ifup

Routing with iptables

With this method, you can connect your guest vm to a tap device in your host. Then you can set iptables rules in your host so that it acts as a router and firewall for your guest.

Routing is done simply by setting the default route on the client to the IP address of the host, allowing IP forwarding, and setting a route to the tap device of the client on the host.


  • Host-side: Allow IPv4 forwarding and add a route to the guest (could be put in a script, but the route has to be added after the guest has started):
sysctl -w net.ipv4.ip_forward=1                 # allow forwarding of IPv4
route add -host <ip-of-client> dev <tap-device> # add route to the client


  • Guest-side: Set the default gateway to the IP address of the host (make sure the host and guest IP addresses are in the same subnet):
route add default gw <ip-of-host>


  • Note: If the host is not on the same subnet as the guest, then you must manually add the route to the host before you create the default route:
route add -host <ip-of-host> dev <network-interface>
route add default gw <ip-of-host>

VDE

Another option is using VDE (Virtual Distributed Ethernet).

More information will be provided later.

Performance

Data on benchmarking results should go in here. There's now a page dedicated to ideas for improving Networking Performance.

Some 10G NIC performance comparisons between VFIO passthrough and virtio are discussed in VFIO vs virtio.

Compatibility

There's another, old and obsolete syntax of specifying network for virtual machines. Above examples uses -netdev..-device model, old way used -net..-net pairs. For example,

-netdev tap,id=net0 -device e1000,netdev=net0,mac=52:54:00:12:34:56

is about the same as old

-net tap,vlan=0 -net nic,vlan=0,model=e1000,macaddr=52:54:00:12:34:56

(note mac => macaddr parameter change as well; vlan=0 is the default).

Old way used the notion of "VLANs" - these are QEMU VLANS, which has nothing to do with 802.1q VLANs. Qemu VLANs are numbered starting with 0, and it's possible to connect one or more devices (either host side, like -net tap, or guest side, like -net nic) to each VLAN, and, in particular, it's possible to connect more than 2 devices to a VLAN. Each device in a VLAN gets all traffic received by every device in it. This model was very confusing for the user (especially when a guest has more than one NIC).

In new model, each host side correspond to just one guest side, forming a pair of devices based on -netdev id= and -device netdev= parameters. It is less confusing, it is faster (because it's always 1:1 pair), and it supports more parameters than old -net..-net way.

However, -net..-net is still supported, used widely, and mentioned in lots of various HOWTOs and guides around the world. It is also a bit shorter and so faster to type.