virttest.staging package¶
Subpackages¶
Submodules¶
virttest.staging.lv_utils module¶
Utilities to create logical volumes or take snapshots of existing ones.
author: | Plamen Dimitrov |
---|---|
copyright: | Intra2net AG 2012 |
license: | GPL v2 |
param vg_name: | Name of the volume group. |
param lv_name: | Name of the logical volume. |
param lv_size: | Size of the logical volume as string in the form “#G” (for example 30G). |
param lv_snapshot_name: | |
Name of the snapshot with origin the logical volume. | |
param lv_snapshot_size: | |
Size of the snapshot with origin the logical volume also as “#G”. | |
param ramdisk_vg_size: | |
Size of the ramdisk virtual group. | |
param ramdisk_basedir: | |
Base directory for the ramdisk sparse file. | |
param ramdisk_sparse_filename: | |
Name of the ramdisk sparse file. |
Sample ramdisk params:
ramdisk_vg_size = "40000"
ramdisk_basedir = "/tmp"
ramdisk_sparse_filename = "virtual_hdd"
Sample general params:
vg_name='autotest_vg',
lv_name='autotest_lv',
lv_size='1G',
lv_snapshot_name='autotest_sn',
lv_snapshot_size='1G'
The ramdisk volume group size is in MB.
virttest.staging.service module¶
-
class
virttest.staging.service.
Factory
[source]¶ Bases:
object
Class to create different kinds of ServiceManager. The all interfaces to create manager are staticmethod, so we do not have to create an instance of factory when create manager.
- GenericServiceManager:
Interface: create_generic_service()
- Description: Object to manage the all services(lldp, sshd and so on).
You can list the all services by GenericServiceManager.list(). And you can operate any service by passing the service name, such as GenericServiceManager.start(“sshd”).
Example: # Get the system service manager service_manager = Factory.create_generic_service()
# Stating service/unit “sshd” service_manager.start(“sshd”)
# Getting a list of available units units = service_manager.list()
- SpecificServiceManager:
interface: create_specific_service(service_name)
- description: Object to manage specific service(such as sshd). You can
not operate the other services nor list the all information on this host.
# Get the specific service manager for sshd sshd = Factory.create_specific_service(“sshd”) sshd.start() sshd.stop()
After all, there is an unified interface to create both of them, create_service(service_name=None).
If we pass a service_name to it, it will return a SpecificServiceManager, otherwise, it will return GenericServiceManager.
-
class
FactoryHelper
(run=<function run>)[source]¶ Bases:
object
Internal class to help create service manager.
Provide some functions to auto detect system type. And auto create command_generator and result_parser.
-
get_generic_service_command_generator
()[source]¶ Lazy initializer for ServiceCommandGenerator using the auto-detect init command.
Returns: ServiceCommandGenerator for the current init command. Return type: _ServiceCommandGenerator
-
get_generic_service_manager_type
()[source]¶ Get the ServiceManager type using the auto-detect init command.
Returns: Subclass type of _GenericServiceManager from the current init command. Return type: _SysVInitServiceManager or _SystemdServiceManager.
-
get_generic_service_result_parser
()[source]¶ Get the ServiceResultParser using the auto-detect init command.
Returns: ServiceResultParser fro the current init command. Return type: _ServiceResultParser
-
get_name_of_init
()[source]¶ Internal function to determine what executable is PID 1, :return: executable name for PID 1, aka init :rtype: str
-
get_specific_service_command_generator
()[source]¶ Create a class that will create partial functions that generate commands for the current init command.
lldpad = SpecificServiceManager("lldpad", auto_create_specific_service_command_generator()) lldpad.start() lldpad.stop()
Returns: A ServiceCommandGenerator for the auto-detected init command. Return type: _ServiceCommandGenerator
-
-
static
Factory.
create_generic_service
(run=<function run>)[source]¶ Detect which init program is being used, init or systemd and return a class with methods to start/stop services.
# Get the system service manager service_manager = Factory.create_generic_service() # Stating service/unit "sshd" service_manager.start("sshd") # Getting a list of available units units = service_manager.list() # Disabling and stopping a list of services services_to_disable = ['ntpd', 'httpd'] for s in services_to_disable: service_manager.disable(s) service_manager.stop(s)
Returns: SysVInitServiceManager or SystemdServiceManager Return type: _GenericServiceManager
-
static
Factory.
create_service
(service_name=None, run=<function run>)[source]¶ # Unified interface for generic and specific service manager.
Returns: _SpecificServiceManager if service_name is not None, _GenericServiceManager if service_name is None.
-
static
Factory.
create_specific_service
(service_name, run=<function run>)[source]¶ # Get the specific service manager for sshd sshd = Factory.create_specific_service(“sshd”) sshd.start() sshd.stop() sshd.reload() sshd.restart() sshd.condrestart() sshd.status() sshd.enable() sshd.disable() sshd.is_enabled()
Parameters: service_name (str) – systemd unit or init.d service to manager Returns: SpecificServiceManager that has start/stop methods Return type: _SpecificServiceManager
-
virttest.staging.service.
convert_systemd_target_to_runlevel
(target)[source]¶ Convert systemd target to runlevel.
Parameters: target (str) – systemd target Returns: sys_v runlevel Return type: str Raises: ValueError – when systemd target is unknown
-
virttest.staging.service.
convert_sysv_runlevel
(level)[source]¶ Convert runlevel to systemd target.
Parameters: level (str or int) – sys_v runlevel Returns: systemd target Return type: str Raises: ValueError – when runlevel is unknown
-
virttest.staging.service.
raw_status_parser
(cmdResult=None)[source]¶ Just return the result of service sub-command.
-
virttest.staging.service.
systemd_command_generator
(command)[source]¶ Generate list of command line argument strings for systemctl. One argument per string for compatibility Popen
WARNING: If systemctl detects that it is running on a tty it will use color, pipe to $PAGER, change column sizes and not truncate unit names. Use –no-pager to suppress pager output, or set PAGER=cat in the environment. You may need to take other steps to suppress color output. See https://bugzilla.redhat.com/show_bug.cgi?id=713567
Parameters: command (str) – start,stop,restart, etc. Returns: list of command and arguments to pass to utils.run or similar functions Return type: list
-
virttest.staging.service.
systemd_list_parser
(cmdResult=None)[source]¶ Parse method for service sub-command list.
:return in form of dict-like, including service name, status and so on
For example:
{"sshd": "enabled", "vsftpd": "disabled", "systemd-sysctl": "static", ... }
-
virttest.staging.service.
systemd_result_parser
(command)[source]¶ Parse results for systemd style commands.
Parameters: command – service sub-command(string). Returns: depends on sub-command.
-
virttest.staging.service.
systemd_status_parser
(cmdResult=None)[source]¶ Parse method for service sub-command status.
:return True : if status is active(running). :return False : if status is stopped. :return None : if status is un-loaded.
-
virttest.staging.service.
sysvinit_command_generator
(command)[source]¶ Generate lists of command arguments for sys_v style inits.
Parameters: command (str) – start,stop,restart, etc. Returns: list of commands to pass to utils.run or similar function Return type: list
-
virttest.staging.service.
sysvinit_list_parser
(cmdResult=None)[source]¶ Parse method for service sub-command list.
:return in form of dict-like, including service name, status and so on
For example:
{"sshd": {0: 'off', 1: 'off', 2: 'off', ..., 6: 'off'}, "vsftpd": {0: 'off', 1: 'off', 2: 'off', ..., 6: 'off'}, "xinetd": {'discard-dgram:': 'off', 'rsync:': 'on',...}, ... }
virttest.staging.utils_cgroup module¶
Helpers for cgroup testing.
copyright: | 2011 Red Hat Inc. |
---|---|
author: | Lukas Doktor <ldoktor@redhat.com> |
-
class
virttest.staging.utils_cgroup.
Cgroup
(module, _client)[source]¶ Bases:
object
Cgroup handling class.
-
cgclassify_cgroup
(pid, cgroup)[source]¶ Classify pid into cgroup
Parameters: - pid – pid of the process
- cgroup – cgroup name
-
cgdelete_cgroup
(cgroup, recursive=False)[source]¶ Delete desired cgroup.
Params cgroup: desired cgroup Params force: If true, sub cgroup can be deleted with parent cgroup
-
cgexec
(cgroup, cmd, args='')[source]¶ Execute command in desired cgroup
Parameters: - cgroup – Desired cgroup
- cmd – Executed command
- args – Executed command’s parameters
-
cgset_property
(prop, value, pwd=None, check=True, checkprop=None)[source]¶ Sets the property value by cgset command
Parameters: - prop – property name (file)
- value – desired value
- pwd – cgroup directory
- check – check the value after setup / override checking value
- checkprop – override prop when checking the value
-
get_cgroup_index
(cgroup)[source]¶ Get cgroup’s index in cgroups
Parameters: cgroup – cgroup name Returns: index of cgroup
-
get_cgroup_name
(pwd=None)[source]¶ Get cgroup’s name
Parameters: pwd – cgroup name Returns: cgroup’s name
-
get_pids
(pwd=None)[source]¶ Get all pids in cgroup
Params: pwd: cgroup directory Returns: all pids(list)
-
get_property
(prop, pwd=None)[source]¶ Gets the property value :param prop: property name (file) :param pwd: cgroup directory :return: [] values or None when FAILED
-
initialize
(modules)[source]¶ Initializes object for use.
Parameters: modules – Array of all available cgroup modules.
-
is_cgroup
(pid, pwd)[source]¶ Checks if the ‘pid’ process is in ‘pwd’ cgroup :param pid: pid of the process :param pwd: cgroup directory :return: 0 when is ‘pwd’ member
-
is_root_cgroup
(pid)[source]¶ Checks if the ‘pid’ process is in root cgroup (WO cgroup) :param pid: pid of the process :return: 0 when is ‘root’ member
-
mk_cgroup
(pwd=None, cgroup=None)[source]¶ Creates new temporary cgroup :param pwd: where to create this cgroup (default: self.root) :param cgroup: desired cgroup name :return: last cgroup index
-
mk_cgroup_cgcreate
(pwd=None, cgroup=None)[source]¶ Make a cgroup by executing the cgcreate command
Params: cgroup: name of the cgroup to be created Returns: last cgroup index
-
set_cgroup
(pid, pwd=None)[source]¶ Sets cgroup membership :param pid: pid of the process :param pwd: cgroup directory
-
set_property
(prop, value, pwd=None, check=True, checkprop=None)[source]¶ Sets the property value :param prop: property name (file) :param value: desired value :param pwd: cgroup directory :param check: check the value after setup / override checking value :param checkprop: override prop when checking the value
-
set_property_h
(prop, value, pwd=None, check=True, checkprop=None)[source]¶ Sets the one-line property value concerning the K,M,G postfix :param prop: property name (file) :param value: desired value :param pwd: cgroup directory :param check: check the value after setup / override checking value :param checkprop: override prop when checking the value
-
-
class
virttest.staging.utils_cgroup.
CgroupModules
(mountdir=None)[source]¶ Bases:
object
Handles the list of different cgroup filesystems.
-
virttest.staging.utils_cgroup.
get_all_controllers
()[source]¶ Get all controllers used in system
Returns: all used controllers(controller_list)
-
virttest.staging.utils_cgroup.
get_cgroup_mountpoint
(controller, mount_file='/proc/mounts')[source]¶ Get desired controller’s mountpoint
Parameters: - controller – Desired controller
- mount_file – Name of file contains mounting information, in most cases this are not need to be set.
Returns: controller’s mountpoint
Raise: TestError when contoller doesn’t exist in mount table
-
virttest.staging.utils_cgroup.
get_load_per_cpu
(_stats=None)[source]¶ Gather load per cpu from /proc/stat :param _stats: previous values :return: list of diff/absolute values of CPU times [SUM, CPU1, CPU2, ...]
-
virttest.staging.utils_cgroup.
resolve_task_cgroup_path
(pid, controller)[source]¶ Resolving cgroup mount path of a particular task
Params: pid : process id of a task for which the cgroup path required Params: controller: takes one of the controller names in controller list Returns: resolved path for cgroup controllers of a given pid
virttest.staging.utils_koji module¶
-
class
virttest.staging.utils_koji.
KojiClient
(cmd=None)[source]¶ Bases:
object
Stablishes a connection with the build system, either koji or brew.
This class provides convenience methods to retrieve information on packages and the packages themselves hosted on the build system. Packages should be specified in the KojiPgkSpec syntax.
-
CMD_LOOKUP_ORDER
= ['/usr/bin/brew', '/usr/bin/koji']¶
-
CONFIG_MAP
= {'/usr/bin/brew': '/etc/brewkoji.conf', '/usr/bin/koji': '/etc/koji.conf'}¶
-
RETRY_STEP
= 3¶
-
RETRY_TIMEOUT
= 30¶
-
get_default_command
()[source]¶ Looks up for koji or brew “binaries” on the system
Systems with plain koji usually don’t have a brew cmd, while systems with koji, have both koji and brew utilities. So we look for brew first, and if found, we consider that the system is configured for brew. If not, we consider this is a system with plain koji.
Returns: either koji or brew command line executable path, or None
-
get_pkg_info
(pkg)[source]¶ Returns information from Koji on the package
Parameters: pkg (KojiPkgSpec) – information about the package, as a KojiPkgSpec instance Returns: information from Koji about the specified package
-
get_pkg_rpm_file_names
(pkg, arch=None)[source]¶ Gets the file names for the RPM packages specified in pkg
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkg_rpm_info
(pkg, arch=None)[source]¶ Returns a list of information on the RPM packages found on koji
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkg_rpm_names
(pkg, arch=None)[source]¶ Gets the names for the RPM packages specified in pkg
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkg_urls
(pkg, arch=None)[source]¶ Gets the urls for the packages specified in pkg
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkgs
(pkg, dst_dir, arch=None)[source]¶ Download the packages
Parameters: - pkg (KojiPkgSpec) – a package specification
- dst_dir (string) – the destination directory, where the downloaded packages will be saved on
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_scratch_pkg_urls
(pkg, arch=None)[source]¶ Gets the urls for the scratch packages specified in pkg
Parameters: - pkg (KojiScratchPkgSpec) – a scratch package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_scratch_pkgs
(pkg, dst_dir, arch=None)[source]¶ Download the packages from a scratch build
Parameters: - pkg (KojiScratchPkgSpec) – a scratch package specification
- dst_dir (string) – the destination directory, where the downloaded packages will be saved on
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_session_options
()[source]¶ Filter only options necessary for setting up a cobbler client session
Returns: only the options used for session setup
-
is_config_valid
()[source]¶ Checks if the currently set koji configuration is valid
Returns: True or False
-
is_pkg_spec_build_valid
(pkg)[source]¶ Checks if build is valid on Koji
Parameters: pkg – a Pkg instance
-
is_pkg_spec_tag_valid
(pkg)[source]¶ Checks if tag is valid on Koji
Parameters: pkg (KojiPkgSpec) – a package specification
-
-
class
virttest.staging.utils_koji.
KojiDirIndexParser
[source]¶ Bases:
HTMLParser.HTMLParser
Parser for HTML directory index pages, specialized to look for RPM links
-
exception
virttest.staging.utils_koji.
KojiDownloadError
(url, timeout, last_error)[source]¶ Bases:
exceptions.IOError
-
class
virttest.staging.utils_koji.
KojiPkgSpec
(text='', tag=None, build=None, package=None, subpackages=[])[source]¶ Bases:
object
A package specification syntax parser for Koji
This holds information on either tag or build, and packages to be fetched from koji and possibly installed (features external do this class).
New objects can be created either by providing information in the textual format or by using the actual parameters for tag, build, package and sub- packages. The textual format is useful for command line interfaces and configuration files, while using parameters is better for using this in a programatic fashion.
The following sets of examples are interchangeable. Specifying all packages part of build number 1000:
>>> from kvm_utils import KojiPkgSpec >>> pkg = KojiPkgSpec('1000')
>>> pkg = KojiPkgSpec(build=1000)
Specifying only a subset of packages of build number 1000:
>>> pkg = KojiPkgSpec('1000:kernel,kernel-devel')
>>> pkg = KojiPkgSpec(build=1000, subpackages=['kernel', 'kernel-devel'])
Specifying the latest build for the ‘kernel’ package tagged with ‘dist-f14’:
>>> pkg = KojiPkgSpec('dist-f14:kernel')
>>> pkg = KojiPkgSpec(tag='dist-f14', package='kernel')
Specifying the ‘kernel’ package using the default tag:
>>> kvm_utils.set_default_koji_tag('dist-f14') >>> pkg = KojiPkgSpec('kernel')
>>> pkg = KojiPkgSpec(package='kernel')
Specifying the ‘kernel’ package using the default tag:
>>> kvm_utils.set_default_koji_tag('dist-f14') >>> pkg = KojiPkgSpec('kernel')
>>> pkg = KojiPkgSpec(package='kernel')
If you do not specify a default tag, and give a package name without an explicit tag, your package specification is considered invalid:
>>> print kvm_utils.get_default_koji_tag() None >>> print kvm_utils.KojiPkgSpec('kernel').is_valid() False
>>> print kvm_utils.KojiPkgSpec(package='kernel').is_valid() False
-
SEP
= ':'¶
-
describe
()[source]¶ Describe this package specification, in a human friendly way
Returns: package specification description
-
is_valid
()[source]¶ Checks if this package specification is valid.
Being valid means that it has enough and not conflicting information. It does not validate that the packages specified actually existe on the Koji server.
Returns: True or False
-
parse
(text)[source]¶ Parses a textual representation of a package specification
Parameters: text (string) – textual representation of a package in koji
-
to_text
()[source]¶ Return the textual representation of this package spec
The output should be consumable by parse() and produce the same package specification.
We find that it’s acceptable to put the currently set default tag as the package explicit tag in the textual definition for completeness.
Returns: package specification in a textual representation
-
-
class
virttest.staging.utils_koji.
KojiScratchPkgSpec
(text='', user=None, task=None, subpackages=[])[source]¶ Bases:
object
A package specification syntax parser for Koji scratch builds
This holds information on user, task and subpackages to be fetched from koji and possibly installed (features external do this class).
New objects can be created either by providing information in the textual format or by using the actual parameters for user, task and subpackages. The textual format is useful for command line interfaces and configuration files, while using parameters is better for using this in a programatic fashion.
This package definition has a special behaviour: if no subpackages are specified, all packages of the chosen architecture (plus noarch packages) will match.
The following sets of examples are interchangeable. Specifying all packages from a scratch build (whose task id is 1000) sent by user jdoe:
>>> from kvm_utils import KojiScratchPkgSpec >>> pkg = KojiScratchPkgSpec('jdoe:1000')
>>> pkg = KojiScratchPkgSpec(user=jdoe, task=1000)
Specifying some packages from a scratch build whose task id is 1000, sent by user jdoe:
>>> pkg = KojiScratchPkgSpec('jdoe:1000:kernel,kernel-devel')
>>> pkg = KojiScratchPkgSpec(user=jdoe, task=1000, subpackages=['kernel', 'kernel-devel'])
-
SEP
= ':'¶
-
-
class
virttest.staging.utils_koji.
RPMFileNameInfo
(filename)[source]¶ Simple parser for RPM based on information present on the filename itself
virttest.staging.utils_memory module¶
-
virttest.staging.utils_memory.
drop_caches
()[source]¶ Writes back all dirty pages to disk and clears all the caches.
-
virttest.staging.utils_memory.
get_buddy_info
(chunk_sizes, nodes='all', zones='all')[source]¶ Get the fragement status of the host. It use the same method to get the page size in buddyinfo. 2^chunk_size * page_size The chunk_sizes can be string make up by all orders that you want to check splited with blank or a mathematical expression with ‘>’, ‘<’ or ‘=’. For example: The input of chunk_size could be: “0 2 4” And the return will be: {‘0’: 3, ‘2’: 286, ‘4’: 687} if you are using expression: “>=9” the return will be: {‘9’: 63, ‘10’: 225}
Parameters: Returns: A dict using the chunk_size as the keys
Return type:
-
virttest.staging.utils_memory.
read_from_numa_maps
(pid, key)[source]¶ Get the process numa related info from numa_maps. This function only use to get the numbers like anon=1.
Parameters: - pid (String) – Process id
- key (String) – The item you want to check from numa_maps
Returns: A dict using the address as the keys
Return type:
-
virttest.staging.utils_memory.
read_from_numastat
(pid, key)[source]¶ Get the process numastat from numastat output.
-
virttest.staging.utils_memory.
read_from_smaps
(pid, key)[source]¶ Get specific item value from the smaps of a process include all sections.
Parameters: - pid (String) – Process id
- key (String) – The item you want to check from smaps
Returns: The value of the item in kb
Return type: