https://linux-kvm.org/api.php?action=feedcontributions&user=Themel&feedformat=atomKVM - User contributions [en]2024-03-29T06:06:41ZUser contributionsMediaWiki 1.39.5https://linux-kvm.org/index.php?title=Migration&diff=2318Migration2009-05-26T08:33:09Z<p>Themel: Removed file:// migration reference - no longer in current kvm.</p>
<hr />
<div>=Migration=<br />
<br />
== Introduction ==<br />
KVM currently supports savevm/loadvm and offline or live migration<br />
Migration commands are given when in qemu-monitor (Alt-Ctrl-2). <br />
Upon successful completion, the migrated VM continues to run on the destination host.<br />
<br />
==== Note ====<br />
You can migrate a guest between an AMD host to an Intel host and back. Naturally, a 64-bit guest can only be migrated to a 64-bit host, but a 32-bit guest can be migrated at will.<br />
<br />
There are some older Intel processors which don't support NX (or XD), which may cause problems in a cluster which includes NX-supporting hosts. To disable NX for a given guest, start it with such a parameter: -cpu qemu64,-nx<br />
<br />
== Requirements ==<br />
* The VM image is accessible on both source and destination hosts (located on a shared storage, e.g. using nfs). <br />
* It is recommended an images-directory would be found on the same path on both hosts (for migrations of a copy-on-write image -- an image created on top of a base-image using "qemu-image create -b ...") <br />
* The src and dst hosts must be on the same subnet (keeping guest's network when tap is used). <br />
* Do not use -snapshot qemu command line option. <br />
* For tcp: migration protocol <br />
* the guest on the destination must be started the same way it was started on the source. <br />
<br />
== highlights / merits ==<br />
* Almost unnoticeable guest down time<br />
* Guest is not involved (unique to KVM Live Migration [#1 1])<br />
* Capability to tunnel VM state through an external program (unique to KVM Live Migration [#1 1])<br />
* ssh/gzip/bzip2/gpg/your own <br />
* Upon success guest continues to run on destination host, upon failure guest continues to run on source host (with one exception) <br />
* Short and Simple <br />
* Easy to enhance <br />
* Hardware independence (almost). <br />
* Support for migration of stopped (paused) VMs. <br />
* Open <br />
<br />
[[Anchor(1)]] 1 These features are unique to KVM Live Migration as far as I know. If you know of other hypervisor that support any of them please update this page or let me (Uri) know. <br />
<br />
== User Interface ==<br />
The user interface is through the qemu monitor (alt-ctrl-2 on the SDL window)<br />
<br />
=== Management ===<br />
migrate [-d] <URI><br />
migrate_cancel <br />
<br />
The '-d' will let you query the status of the migration.<br />
<br />
With no '-d' the monitor prompt returns when the migration completes.<br />
URI can be one of 'exec:<command>' or tcp:<ip:port> <br />
<br />
=== Status ===<br />
<br />
info migrate <br />
<br />
=== Migration Parameters ===<br />
migrate_set_speed <speed> set bandwidth control parameters (max transfer rate per second)<br />
<br />
== Example / HOWTO ==<br />
A is the source host, B is the destination host:<br />
===''TCP'' example:===<br />
1. Start the VM on B with the exact same parameters as the VM on A, in migration-listen mode (-incoming tcp:0:4444 (or other PORT)) <br />
2. Start the migration (always on the source host): <br />
A: migrate -d tcp:B:4444 (or other PORT) <br />
3. Check the status (on A only): <br />
info migrate <br />
<br />
===''Offline'' example:===<br />
1. unlimit bandwidth used for migration:<br />
A: migrate_set_speed 1g <br />
2. stop the guest: <br />
A: stop <br />
3. continue with either TCP or exec migration as described above.<br />
<br />
== Problems / Todo ==<br />
* TSC offset on the new host must be set in such a way that the guest sees a monotonically increasing TSC, otherwise the guest may hang indefinitely after migration. <br />
* usbdevice tablet complains after migration. <br />
* handle migration completion better (especially when network problems occur). <br />
* More informative status. <br />
* Migration does not work while CPU real-mode/protected mode are still changing. <br />
<br />
== savevm/loadvm to an external state file (using pseudo-migration) ==<br />
<br />
* To be supported directly by Migration Protocols, but until then...<br />
* Save VM state into a compressed file <br />
** Save <br />
stop <br />
migrate_set_speed 4095m <br />
migrate "exec:gzip -c > STATEFILE.gz" <br />
<br />
** Load<br />
gzip -c -d STATEFILE.gz | <qemu-command-line> -incoming stdio<br />
* Save VM State into an encrypted file (assumes KEY has already been generated)<br />
** Save <br />
stop <br />
migrate_set_speed 4095m <br />
migrate "exec:gpg -q -e -r KEY -o STATFILE.gpg"<br />
<br />
* Load VM state from an encrypted file<br />
gpg -q -d -r KEY STATEFILE.gpg | <qemu-command-line> -incoming stdio<br />
<br />
== more exec: options ==<br />
<br />
* Send encrypted VM state from host A to host B (Note: ssh is probably better, this is just for show)<br />
<br />
** on host A<br />
migrate "exec:gpg -q -e -r KEY | nc B 4444"<br />
<br />
** on host B<br />
nc -l 4444 | gpg -q -d -r KEY | <qemu-command-line> -incoming stdio<br />
<br />
== Algorithm (the short version) ==<br />
1. Setup<br />
* Start guest on destination, connect, enable dirty page logging and more<br />
2. Transfer Memory<br />
* Guest continues to run<br />
* Bandwidth limitation (controlled by the user)<br />
* First transfer the whole memory<br />
* Iteratively transfer all dirty pages (pages that were written to by the guest).<br />
3. Stop the guest<br />
* And sync VM image(s) (guest's hard drives).<br />
4. Transfer State<br />
* As fast as possible (no bandwidth limitation)<br />
* All VM devices' state and dirty pages yet to be transferred<br />
5. Continue the guest<br />
* On destination upon success<br />
** Broadcast "I'm over here" Ethernet packet to announce new location of NIC(s).<br />
* On source upon failure (with one exception).<br />
<br />
<br />
Instructions for kvm-13 and below: [[MigrationQemu0.8.2]].</div>Themel