KVM Autotest Refactor page

From KVM
Revision as of 21:09, 7 December 2010 by Lmr (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

KVM autotest refactor

Suggested approach

   1) Move code that has negligible impact [see table below] from 'client/tests/kvm' to 'client' ASAP
     + This would immediatelly benefit xen autotest, since it is frequently lagging behing features and fixes. One example is newer syntax in kvm_config.py that was not available in the xen autotest counterpart.
   2) Synchronize that code (step 1) usage between kvm and xen autotest
     + Rewrite current tests, changing imports of modules from 'kvm_*' and 'xen_' to a common prefix ('virt_*' ?).
     + This can be done by an automated script and verified by the unit tests.
   3) Work gradually on code that would generate more impact, implementing abstractions and other mechanisms
     + The config file and thus the params passed to tests already add a lot of flexibility [see example snippet #1]
     + Improve monitor functionality, introducing methods that wrap the hypervisor instead of sending specific commands [see example snippet #2, #3]. 
     + This would eventually become a rather large API. libvirt API should be evaluated and if considered sound and appropriate for autotest, could be made similar.
     + Modify tests so that they make use of improved Monitor/VM API
   4) Publish resulting API and best practices documentation so that new tests writers may submit 


Other notes

  * Regarding the prefix (virt_* ?), some functionality is not even tied to virtualization (such as the configuration format/parser) and could benefit other complex test modules;
  * kvm_vm:
   - There are significant differences in (KVM) VM class and (xen) XendDomain. Methods such as create() are quite different in parameters in implementation
   - (KVM) VM uses a 'monitor' to interact with qemu, whether (xen) XenDomain executes 'xm' commands 
   - the 'monitor' abstraction is valid, but should have clear boundaries 
   - xen autotest should also get a 'monitor' implementation
   - this even paves the way to a libvirt based monitor implementation
   - probably a capabilities mechanism would be necessary to acomodate diferences between virt technologies (see kvm_monitor below)
 * kvm_monitor
  - There are methods and parameters which do not map completely to Xen, specially PV guests (screendump, migrate with full disk copy, mouse move, etc)
  - Again, a capabilities mechanism could be used


Example #1 (pci_hotplug.py)

Parameters provide a lot of flexibility and diminish the need to implement a interface on a variant that does not need or cannot provide it. This code snippet:

26 # Modprobe the module if specified in config file 27 module = params.get("modprobe_module") 28 if module: 29 session.cmd("modprobe %s" % module)

could be kept exactly like that, loading 'acpiphp' as it does now for RHEL guests onon kvm and nothing for RHEL xen domU kernels (with XEN_PCIDEV_FRONTEND=y).

If someone wants to test a xen domU kernel with modular XEN_PCIDEV_FRONTEND, only this parameter would have to be adjusted.


Example #2 (pci_hotplug.py)

Only operations from the monitor side (vm.monitor.*) would have to be implemented for each virtualization technology. On the guest side (session.cmd_output() and the like), things could go unchanged, helped by parameters when necessary.

Curent code:

31 # Get output of command 'info pci' as reference 32 info_pci_ref = vm.monitor.info("pci") 33 34 # Get output of command as reference 35 reference = session.cmd_output(params.get("reference_cmd"))

Suggested code:

31 # Get output of command 'info pci' as reference 32 info_pci_ref = vm.monitor.get_pci_info() 33 34 # Get output of command as reference 35 reference = session.cmd_output(params.get("reference_cmd"))


Example #3 (pci_hotplug.py)

Current test code not only interacts directly with qemu but also deals with different versions and syntaxes:

41 # Probe qemu to verify what is the supported syntax for PCI hotplug 42 cmd_output = vm.monitor.cmd("?") 43 if len(re.findall("\ndevice_add", cmd_output)) > 0: 44 cmd_type = "device_add" 45 elif len(re.findall("\npci_add", cmd_output)) > 0: 46 cmd_type = "pci_add" 47 else: 48 raise error.TestError("Unknow version of qemu")

Again, this should be abstracted in the monitor implementation, and consumed by the test code (this is even commented in the current code):

59 # Execute pci_add (should be replaced by a proper monitor method call) 60 add_output = vm.monitor.cmd(pci_add_cmd) 61 if not "OK domain" in add_output: 62 raise error.TestFail("Add PCI device failed. " 63 "Monitor command is: %s, Output: %r" % 64 (pci_add_cmd, add_output)) 65 after_add = vm.monitor.info("pci")

This should become:

59 # Execute pci_add 60 if not vm.monitor_pci_add(model): 61 raise error.TestFail("Add PCI device failed.") 62 after_add = vm.monitor.get_pci_info()


Library code under client/tests/kvm

The componentes are sorted by depencies, so that working down the list would solve all depencies, except when circular dependencies exist.

Filename Impact Dependencies Spec Tied Xen


------ ------------ ---- ---- ---

kmv_config.py Negligible kvm_utils (logging) No No Yes kvm_subprocess.py Negligible none No No Yes scan_results.py Negligible none No No Yes cd_hash.py Negligible kvm_utils (logging) No No No rss_file_transfer.py Negligible none No No No

kvm_monitor.py High kvm_utils (random_str) No Yes No[1] kvm_vm.py High kvm_utils No Yes Yes kvm_subprocess kvm_monitor rss_file_transfer

kvm_scheduler.py Medium kvm_utils No[2] Yes[3] No kvm_preprocessing.py Medium kvm_vm No Yes[4] Yes kvm_utils kvm_subprocess kvm_monitor kvm_utils.py High kvm_subprocess No Yes[5] Yes

kvm_test_utils.py High kvm_utils No Yes Yes kvm_vm kvm_subprocess scan_results

Legend:


- Spec: Specific for KVM (that is, not applicable to other virt technologies) - Tied: Currently tied to KVM in its current implementation - Xen: Currently used by xen autotest or has equivalent/derived implementation

[1] - Xen autotest currently does not implement a monitor, but this analysis recommends one to be implemented [2] - Currently xen autotest does not have a control file that uses parallel tests, so it's low priority but could be useful [3] - Tied only to kvm.VM and vm.monitor, used as abstractions [4] - Tied only to kvm.VM and vm.monitor, used as abstractions [5] - There's both kvm specific code and code that should be shared between xen and kvm autotest