KVM-Autotest-Docs: Difference between revisions

From KVM
No edit summary
No edit summary
Line 1: Line 1:
== Introduction ==
== Introduction ==


Recent autotest supports passing test parameters to control file through --args options. This is also been introduced in kvm-autotest which can let user do some basic tests configurations (variants limitation or excluding, parameters configuration) through command line.
Recent autotest supports passing test parameters to control file through --args options. This is also been introduced in kvm-autotest which can let user do some basic tests configurations (variants limitation/excluding, parameters configuration) through command line.


You should have some basic understanding of the test parameters and configuration file format before using the command line. If not, you are suggested to read the [[KVM-Autotest/Test Config File|test configuration file format]] and [[KVM-Autotest/Parameters|test parameters]] before continuing.
You should have some basic understanding of the test parameters and configuration file format before using the command line. If not, you are suggested to read the [[KVM-Autotest/Test Config File|test configuration file format]] and [[KVM-Autotest/Parameters|test parameters]] before continuing.
Line 13: Line 13:
KVM-Autotest supports two kinds of parameters:
KVM-Autotest supports two kinds of parameters:


# keywords "only" and "no": they serve as the same function as in configuration file, limit and exclude variants.
# keywords "only" and "no": they serve the same function as in configuration file, limit and exclude variants.
## For each "only=x", the control file would generate a "only x" and pass it to the configuration file analyzer
## For each "only=x", the control file would generate a "only x" and pass it to the configuration file analyzer
## For each "no=y", the control would generate a "no y" and pass it to the configuration file analyzer
## For each "no=y", the control would generate a "no y" and pass it to the configuration file analyzer
Line 22: Line 22:
The command line parameters interface can not provide the same flexibility as the configuration file and have the following limitations:
The command line parameters interface can not provide the same flexibility as the configuration file and have the following limitations:


# The generation of the final test matrix still depends on the configuration file, you still need it as a base and do test. You still need a test template such as tests.cfg in order to utilize the command line interface.
# The generation of the final test matrix still depends on the configuration file, you still need it as a base to do the test.  
# As all test parameters passed from the command line and analyzed after the configuration file, so any test parameters would override ALL previous ones defined in configuration file. So the command line interface is good at being used to do small configuration modifications to just one test.
# As all test parameters passed from the command line and analyzed after the configuration file, any test parameters passed from cmdline would override ALL previous ones defined in configuration file. So the command line interface is better to be used to specify the frequently changed parameters, for the parameter which is not changed frequently, it's suggested to put them as a customized variant in configuration file.
# It can not be used to adding variants, if you want them, you must edit the configuration file.
# It can not be used to adding variants, if you want them, you must edit the configuration file.


== Examples ==
== Examples ==


Suppose you use the default tests.cfg.sample as the tests.cfg. The first step is comment the last line
Suppose you use the default tests.cfg.sample as the tests.cfg. The first step is comment the last line as it has greatly reduced the test matrix which would block our configuration.


As the default test matrix too huge to run, so the second thing is to know the eight kinds of variants which could be used to reduce the matrix.
As the default test matrix too huge to run, so the second thing is to know the eight kinds of variants which could be used to reduce the matrix.


# Operating Systems: We must specify the type of operating system of the image through which KVM-Autoest can decide the os-dependent parameters such as shell client and others. You can take a look at test_base.cfg.sample to get the accurate names.
# Operating Systems: We must specify the type of operating system of the image through which KVM-Autoest can decide the os-dependent parameters such as the type of shell client and others. You can take a look at test_base.cfg.sample to get the accurate names.
# Test cases: The test case you want to run, see test_base.cfg.sample for details
# Test cases: The test case you want to run, see test_base.cfg.sample for details
# Type of virtual network card: For example, you can use "only=e1000" in cmdline to test e1000 only. For expert, this can be overrided by "nic_model=XXX"
# Type of virtual network card: For example, you can use "only=e1000" in cmdline to test e1000 only. For expert, this can be overrided by "nic_model=XXX"
Line 57: Line 57:
## variant name: hugepages: Use hugepages
## variant name: hugepages: Use hugepages


Only after the you've choosing the above eight variants (either through configuration file or cmdline or both) can you get just one test. For the variants which is not modified frequently, you'd better to put them in the configuration file.
Only after the you've chose the above eight variants (either through configuration file or cmdline or both) can you get one test. For the variants which is not modified frequently, you'd better to put them in the configuration file.


If I have a pre-installed image (RHEL.5.5.x86_64 locate at /tmp/x.qcow2 with password XYZ, and I want to run a reboot test on it, how can I do this through command line?
If I have a pre-installed image (RHEL.5.5.x86_64 locate at /tmp/x.qcow2 with password XYZ, and I want to run a reboot test on it, how can I do this through command line?
Line 68: Line 68:
|-
|-
| only=full
| only=full
| Use the variant full in tests.cfg which contains the biggest test matrix, this would let us to choose our desired at will. You can put the unmodified parameters/variant in a customized variant and just use it through cmdline.
| Use the variant full in tests.cfg which contains the biggest test matrix, this would let us to choose our desired configuration at will. You can put the parameters/variant which is not changed frequently in a customized variant and just use it through cmdline.
|-
|-
| qemu_binary=/usr/bin/qemu-kvm
| qemu_binary=/usr/bin/qemu-kvm
| Specify the path of the qemu-kvm binary. If this is fixed, it's better to put this in you customized variants.
| Specify the path of the qemu-kvm binary. If this path is fixed or you do not want to test other version of qemu, it's better to put this in you customized variants.
|-
|-
| only=boot
| only=boot
| Only test booting.
| Choose the testcase, only boot is tested
|-
|-
| only=smallpages
| only=smallpages
Line 80: Line 80:
|-
|-
| only=ide
| only=ide
| Only use IDE drive.
| Choose the disk type, only use IDE drive.
|-  
|-  
| image_name=/tmp/x only=qcow2
| image_name=/tmp/x only=qcow2
| Use /tmp/x.qcow2 as the disk of guest
| Specify the image name, use /tmp/x.qcow2 as the disk of guest
|-
|-
| only=no_pci_assignable
| only=no_pci_assignable
Line 89: Line 89:
|-
|-
| only=RHEL.5.5.x86_64  
| only=RHEL.5.5.x86_64  
| Guest is a 64bit RHEL5.5.
| Choose the type of guest, 64bit RHEL5.5 is used.
|-
|-
| password=XYZ
| password=XYZ
| Root password is XYZ
| Override the default password.
|}
|}



Revision as of 02:49, 24 January 2011

Introduction

Recent autotest supports passing test parameters to control file through --args options. This is also been introduced in kvm-autotest which can let user do some basic tests configurations (variants limitation/excluding, parameters configuration) through command line.

You should have some basic understanding of the test parameters and configuration file format before using the command line. If not, you are suggested to read the test configuration file format and test parameters before continuing.

How to use

It's very simple for you to use the command line, suppose you are in the directory of kvm subtest, you can pass test parameters through:

../../bin/autotest control --args="key1=value1 key2=value2 ...... keyN=valueN"

KVM-Autotest supports two kinds of parameters:

  1. keywords "only" and "no": they serve the same function as in configuration file, limit and exclude variants.
    1. For each "only=x", the control file would generate a "only x" and pass it to the configuration file analyzer
    2. For each "no=y", the control would generate a "no y" and pass it to the configuration file analyzer
  2. test parameters: all other are passed to the configuration analyzer untouched, that is to say, you could specify any test parameters by this way.

Limitations

The command line parameters interface can not provide the same flexibility as the configuration file and have the following limitations:

  1. The generation of the final test matrix still depends on the configuration file, you still need it as a base to do the test.
  2. As all test parameters passed from the command line and analyzed after the configuration file, any test parameters passed from cmdline would override ALL previous ones defined in configuration file. So the command line interface is better to be used to specify the frequently changed parameters, for the parameter which is not changed frequently, it's suggested to put them as a customized variant in configuration file.
  3. It can not be used to adding variants, if you want them, you must edit the configuration file.

Examples

Suppose you use the default tests.cfg.sample as the tests.cfg. The first step is comment the last line as it has greatly reduced the test matrix which would block our configuration.

As the default test matrix too huge to run, so the second thing is to know the eight kinds of variants which could be used to reduce the matrix.

  1. Operating Systems: We must specify the type of operating system of the image through which KVM-Autoest can decide the os-dependent parameters such as the type of shell client and others. You can take a look at test_base.cfg.sample to get the accurate names.
  2. Test cases: The test case you want to run, see test_base.cfg.sample for details
  3. Type of virtual network card: For example, you can use "only=e1000" in cmdline to test e1000 only. For expert, this can be overrided by "nic_model=XXX"
    1. variant name: rtl8139: Realtek 8139C
    2. variant name: virtio_net: Virtio Network Adapter
    3. variant name: e1000: Inter e1000 Adapter
  4. Type of block device: For example, you can use "only=e1000" in cmdline to test e1000 only. For expert, this can be overrided by "nic_model=XXX"
    1. variant name: ide: IDE drive
    2. variant name: scsi: SCSI driver
    3. variant name: virtio_blk: Virtio Block Drive
  5. Type of image format: For example, you can use "only=qcow2" to test qcow2 only, and if you want to test a pre-installed image XXX.qcow2, you should use "image_name=XXX only=qcow2". For expert, you can use "image_format=XXX" to override.
    1. variant name: qcow2: Qemu Copy-On-Write Version 2 image format
    2. variant name: raw: RAW format
    3. variant name: vmdk: VMWare Virtual Machine DisK image format
  6. Num of vcpus, For example, you can use "only=smp2" to test 2 vcpus, and if you want to test more vcpus you can use "smp=$num_of_cpus"
    1. variant name: up: Only use one vcpu
    2. variant name: smp2: Use two vcpus
  7. PCI Assignable Status, For example, you can use "only=no_pci_assignable" to test ordinary scenario. For expert, you can override this through "pci_assignable=XX"
    1. variant name: no_pci_assignalbe: Do not use device assignment in the test
    2. variant name: pf_assignable: Assign a pf to guest
    3. variant name: vf_assignable: Assign a vf to guest
  8. Whether use hugepages
    1. variant name: smallpages: Do not use hugepages
    2. variant name: hugepages: Use hugepages

Only after the you've chose the above eight variants (either through configuration file or cmdline or both) can you get one test. For the variants which is not modified frequently, you'd better to put them in the configuration file.

If I have a pre-installed image (RHEL.5.5.x86_64 locate at /tmp/x.qcow2 with password XYZ, and I want to run a reboot test on it, how can I do this through command line?

../../bin/autotest control --verbose --args="only=full qemu_binary=/usr/bin/qemu-kvm only=boot only=rtl8139 only=smallpages only=ide image_name=/tmp/x only=qcow2 only=smp2 only=no_pci_assignable only=RHEL.5.5.x86_64 password=XYZ"

params meaning
only=full Use the variant full in tests.cfg which contains the biggest test matrix, this would let us to choose our desired configuration at will. You can put the parameters/variant which is not changed frequently in a customized variant and just use it through cmdline.
qemu_binary=/usr/bin/qemu-kvm Specify the path of the qemu-kvm binary. If this path is fixed or you do not want to test other version of qemu, it's better to put this in you customized variants.
only=boot Choose the testcase, only boot is tested
only=smallpages Do not allocate hugepages for the guest.
only=ide Choose the disk type, only use IDE drive.
image_name=/tmp/x only=qcow2 Specify the image name, use /tmp/x.qcow2 as the disk of guest
only=no_pci_assignable Run guest without any assigned devices.
only=RHEL.5.5.x86_64 Choose the type of guest, 64bit RHEL5.5 is used.
password=XYZ Override the default password.

TODO

  1. support parameters surrounded by quotes
  2. cli support for build?
  3. auto completion?