KVM Autotest Refactor page

From KVM
Revision as of 21:09, 7 December 2010 by Lmr (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

KVM autotest refactor

Suggested approach

   1) Move code that has negligible impact [see table below] from 'client/tests/kvm' to 'client' ASAP
     + This would immediatelly benefit xen autotest, since it is frequently lagging behing features and fixes. One example is newer syntax in kvm_config.py that was not available in the xen autotest counterpart.
   2) Synchronize that code (step 1) usage between kvm and xen autotest
     + Rewrite current tests, changing imports of modules from 'kvm_*' and 'xen_' to a common prefix ('virt_*' ?).
     + This can be done by an automated script and verified by the unit tests.
   3) Work gradually on code that would generate more impact, implementing abstractions and other mechanisms
     + The config file and thus the params passed to tests already add a lot of flexibility [see example snippet #1]
     + Improve monitor functionality, introducing methods that wrap the hypervisor instead of sending specific commands [see example snippet #2, #3]. 
     + This would eventually become a rather large API. libvirt API should be evaluated and if considered sound and appropriate for autotest, could be made similar.
     + Modify tests so that they make use of improved Monitor/VM API
   4) Publish resulting API and best practices documentation so that new tests writers may submit 


Other notes

  * Regarding the prefix (virt_* ?), some functionality is not even tied to virtualization (such as the configuration format/parser) and could benefit other complex test modules;
  * kvm_vm:
   - There are significant differences in (KVM) VM class and (xen) XendDomain. Methods such as create() are quite different in parameters in implementation
   - (KVM) VM uses a 'monitor' to interact with qemu, whether (xen) XenDomain executes 'xm' commands 
   - the 'monitor' abstraction is valid, but should have clear boundaries 
   - xen autotest should also get a 'monitor' implementation
   - this even paves the way to a libvirt based monitor implementation
   - probably a capabilities mechanism would be necessary to acomodate diferences between virt technologies (see kvm_monitor below)
 * kvm_monitor
  - There are methods and parameters which do not map completely to Xen, specially PV guests (screendump, migrate with full disk copy, mouse move, etc)
  - Again, a capabilities mechanism could be used


Example #1 (pci_hotplug.py)

Parameters provide a lot of flexibility and diminish the need to implement a interface on a variant that does not need or cannot provide it. This code snippet:

26 # Modprobe the module if specified in config file 27 module = params.get("modprobe_module") 28 if module: 29 session.cmd("modprobe %s" % module)

could be kept exactly like that, loading 'acpiphp' as it does now for RHEL guests onon kvm and nothing for RHEL xen domU kernels (with XEN_PCIDEV_FRONTEND=y).

If someone wants to test a xen domU kernel with modular XEN_PCIDEV_FRONTEND, only this parameter would have to be adjusted.


Example #2 (pci_hotplug.py)

Only operations from the monitor side (vm.monitor.*) would have to be implemented for each virtualization technology. On the guest side (session.cmd_output() and the like), things could go unchanged, helped by parameters when necessary.

Curent code:

31 # Get output of command 'info pci' as reference 32 info_pci_ref = vm.monitor.info("pci") 33 34 # Get output of command as reference 35 reference = session.cmd_output(params.get("reference_cmd"))

Suggested code:

31 # Get output of command 'info pci' as reference 32 info_pci_ref = vm.monitor.get_pci_info() 33 34 # Get output of command as reference 35 reference = session.cmd_output(params.get("reference_cmd"))


Example #3 (pci_hotplug.py)

Current test code not only interacts directly with qemu but also deals with different versions and syntaxes:

41 # Probe qemu to verify what is the supported syntax for PCI hotplug 42 cmd_output = vm.monitor.cmd("?") 43 if len(re.findall("\ndevice_add", cmd_output)) > 0: 44 cmd_type = "device_add" 45 elif len(re.findall("\npci_add", cmd_output)) > 0: 46 cmd_type = "pci_add" 47 else: 48 raise error.TestError("Unknow version of qemu")

Again, this should be abstracted in the monitor implementation, and consumed by the test code (this is even commented in the current code):

59 # Execute pci_add (should be replaced by a proper monitor method call) 60 add_output = vm.monitor.cmd(pci_add_cmd) 61 if not "OK domain" in add_output: 62 raise error.TestFail("Add PCI device failed. " 63 "Monitor command is: %s, Output: %r" % 64 (pci_add_cmd, add_output)) 65 after_add = vm.monitor.info("pci")

This should become:

59 # Execute pci_add 60 if not vm.monitor_pci_add(model): 61 raise error.TestFail("Add PCI device failed.") 62 after_add = vm.monitor.get_pci_info()


Library code under client/tests/kvm

The componentes are sorted by depencies, so that working down the list would solve all depencies, except when circular dependencies exist.

Filename Impact Dependencies Spec Tied Xen


------ ------------ ---- ---- ---

kmv_config.py Negligible kvm_utils (logging) No No Yes kvm_subprocess.py Negligible none No No Yes scan_results.py Negligible none No No Yes cd_hash.py Negligible kvm_utils (logging) No No No rss_file_transfer.py Negligible none No No No

kvm_monitor.py High kvm_utils (random_str) No Yes No[1] kvm_vm.py High kvm_utils No Yes Yes kvm_subprocess kvm_monitor rss_file_transfer

kvm_scheduler.py Medium kvm_utils No[2] Yes[3] No kvm_preprocessing.py Medium kvm_vm No Yes[4] Yes kvm_utils kvm_subprocess kvm_monitor kvm_utils.py High kvm_subprocess No Yes[5] Yes

kvm_test_utils.py High kvm_utils No Yes Yes kvm_vm kvm_subprocess scan_results

Legend:


- Spec: Specific for KVM (that is, not applicable to other virt technologies) - Tied: Currently tied to KVM in its current implementation - Xen: Currently used by xen autotest or has equivalent/derived implementation

[1] - Xen autotest currently does not implement a monitor, but this analysis recommends one to be implemented [2] - Currently xen autotest does not have a control file that uses parallel tests, so it's low priority but could be useful [3] - Tied only to kvm.VM and vm.monitor, used as abstractions [4] - Tied only to kvm.VM and vm.monitor, used as abstractions [5] - There's both kvm specific code and code that should be shared between xen and kvm autotest