Lmr at 01:16, 8 December 2010

2010-12-08T01:16:02Z

Lmr at 01:09, 8 December 2010

2010-12-08T01:09:11Z

New page

= KVM autotest refactor =

== Suggested approach ==

1) Move code that has negligible impact [see table below] from 'client/tests/kvm' to 'client' ASAP
+ This would immediatelly benefit xen autotest, since it is frequently lagging behing features and fixes. One example is newer syntax in kvm_config.py that was not available in the xen autotest counterpart.

2) Synchronize that code (step 1) usage between kvm and xen autotest
+ Rewrite current tests, changing imports of modules from 'kvm_*' and 'xen_' to a common prefix ('virt_*' ?).
+ This can be done by an automated script and verified by the unit tests.

3) Work gradually on code that would generate more impact, implementing abstractions and other mechanisms
+ The config file and thus the params passed to tests already add a lot of flexibility [see example snippet #1]
+ Improve monitor functionality, introducing methods that wrap the hypervisor instead of sending specific commands [see example snippet #2, #3].
+ This would eventually become a rather large API. libvirt API should be evaluated and if considered sound and appropriate for autotest, could be made similar.
+ Modify tests so that they make use of improved Monitor/VM API

4) Publish resulting API and best practices documentation so that new tests writers may submit

== Other notes ==

* Regarding the prefix (virt_* ?), some functionality is not even tied to virtualization (such as the configuration format/parser) and could benefit other complex test modules;

* kvm_vm:
- There are significant differences in (KVM) VM class and (xen) XendDomain. Methods such as create() are quite different in parameters in implementation
- (KVM) VM uses a 'monitor' to interact with qemu, whether (xen) XenDomain executes 'xm' commands
- the 'monitor' abstraction is valid, but should have clear boundaries
- xen autotest should also get a 'monitor' implementation
- this even paves the way to a libvirt based monitor implementation
- probably a capabilities mechanism would be necessary to acomodate diferences between virt technologies (see kvm_monitor below)

* kvm_monitor
- There are methods and parameters which do not map completely to Xen, specially PV guests (screendump, migrate with full disk copy, mouse move, etc)
- Again, a capabilities mechanism could be used

== Example #1 (pci_hotplug.py) ==

Parameters provide a lot of flexibility and diminish the need to implement a interface on a variant that does not need or cannot provide it. This code snippet:

26 # Modprobe the module if specified in config file
27 module = params.get("modprobe_module")
28 if module:
29 session.cmd("modprobe %s" % module)

could be kept exactly like that, loading 'acpiphp' as it does now for RHEL guests onon kvm and nothing for RHEL xen domU kernels (with XEN_PCIDEV_FRONTEND=y).

If someone wants to test a xen domU kernel with modular XEN_PCIDEV_FRONTEND, only this parameter would have to be adjusted.

== Example #2 (pci_hotplug.py) ==

Only operations from the monitor side (vm.monitor.*) would have to be implemented for each virtualization technology. On the guest side (session.cmd_output() and the like), things could go unchanged, helped by parameters when necessary.

Curent code:

31 # Get output of command 'info pci' as reference
32 info_pci_ref = vm.monitor.info("pci")
33
34 # Get output of command as reference
35 reference = session.cmd_output(params.get("reference_cmd"))

Suggested code:

31 # Get output of command 'info pci' as reference
32 info_pci_ref = vm.monitor.get_pci_info()
33
34 # Get output of command as reference
35 reference = session.cmd_output(params.get("reference_cmd"))

== Example #3 (pci_hotplug.py) ==

Current test code not only interacts directly with qemu but also deals with different versions and syntaxes:

41 # Probe qemu to verify what is the supported syntax for PCI hotplug
42 cmd_output = vm.monitor.cmd("?")
43 if len(re.findall("\ndevice_add", cmd_output)) > 0:
44 cmd_type = "device_add"
45 elif len(re.findall("\npci_add", cmd_output)) > 0:
46 cmd_type = "pci_add"
47 else:
48 raise error.TestError("Unknow version of qemu")

Again, this should be abstracted in the monitor implementation, and consumed by the test code (this is even commented in the current code):

59 # Execute pci_add (should be replaced by a proper monitor method call)
60 add_output = vm.monitor.cmd(pci_add_cmd)
61 if not "OK domain" in add_output:
62 raise error.TestFail("Add PCI device failed. "
63 "Monitor command is: %s, Output: %r" %
64 (pci_add_cmd, add_output))
65 after_add = vm.monitor.info("pci")

This should become:

59 # Execute pci_add
60 if not vm.monitor_pci_add(model):
61 raise error.TestFail("Add PCI device failed.")
62 after_add = vm.monitor.get_pci_info()

== Library code under client/tests/kvm ==

The componentes are sorted by depencies, so that working down the list would solve all depencies, except when circular dependencies exist.

Filename Impact Dependencies Spec Tied Xen
-------- ------ ------------ ---- ---- ---
kmv_config.py Negligible kvm_utils (logging) No No Yes
kvm_subprocess.py Negligible none No No Yes
scan_results.py Negligible none No No Yes
cd_hash.py Negligible kvm_utils (logging) No No No
rss_file_transfer.py Negligible none No No No

kvm_monitor.py High kvm_utils (random_str) No Yes No[1]
kvm_vm.py High kvm_utils No Yes Yes
kvm_subprocess
kvm_monitor
rss_file_transfer

kvm_scheduler.py Medium kvm_utils No[2] Yes[3] No
kvm_preprocessing.py Medium kvm_vm No Yes[4] Yes
kvm_utils
kvm_subprocess
kvm_monitor
kvm_utils.py High kvm_subprocess No Yes[5] Yes

kvm_test_utils.py High kvm_utils No Yes Yes
kvm_vm
kvm_subprocess
scan_results

Legend:
-------
- Spec: Specific for KVM (that is, not applicable to other virt technologies)
- Tied: Currently tied to KVM in its current implementation
- Xen: Currently used by xen autotest or has equivalent/derived implementation

[1] - Xen autotest currently does not implement a monitor, but this analysis recommends one to be implemented
[2] - Currently xen autotest does not have a control file that uses parallel tests, so it's low priority but could be useful
[3] - Tied only to kvm.VM and vm.monitor, used as abstractions
[4] - Tied only to kvm.VM and vm.monitor, used as abstractions
[5] - There's both kvm specific code and code that should be shared between xen and kvm autotest

KVM Autotest Refactor page - Revision history

Lmr at 01:16, 8 December 2010

Lmr at 01:09, 8 December 2010