Sign Up
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
systemsmanagement:Uyuni:Snapshots:2021.08
salt
opensuse-3000.2-virt-backports-236-257.patch
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File opensuse-3000.2-virt-backports-236-257.patch of Package salt
From 895231d5605e63882de015d7b99cea36e9174d66 Mon Sep 17 00:00:00 2001 From: Cedric Bosdonnat <cbosdonnat@suse.com> Date: Tue, 1 Sep 2020 10:38:39 +0200 Subject: [PATCH] Opensuse 3000.2 virt backports (#236) (#257) * Revert "virt._get_domain: don't raise an exception if there is no VM" This reverts commit 9d433776fc252ab872b331cfa7fc940c42452f83. * Revert "openSUSE-3000 virt-defined-states (#222)" This reverts commit 900fdc4d5bcfdac933f9a411d38b92ef47dd1ef5. * Revert "Add virt.all_capabilities" This reverts commit 96e8307b531f7e51f5587f66a51dc52ad246c8c1. * Blacken salt -- virt only * virt._get_domain: don't raise an exception if there is no VM Raising an exception if there is no VM in _get_domain makes sense if looking for some VMs, but not when listing all VMs. * fixed bug on update existing boot parameters * add support to boot vm with UEFI * virt: don't depend on ElementTree.tostring to compare XML fragments ElementTree.tostring() implementation starts to keep the attribute order defined by the user in python 3.8. This can lead to equal elements to be considered different. Use xmlutil.to_dict(element, True) to compare the elements. This also uncovered a wrong behavior of pool_update when only changing the password. * openSUSE-3000 virt-defined-states (#222) * Create virt.pool_defined state out of virt.pool_running Users may want to use states to ensure a virtual storage pool is defined and not enforce it to be running. Extract the code that performs the pool definition / update from virt.pool_running state into a virt.pool_defined. Obviously the virt.pool_running state calls the virt.pool_defined one. In such a case no additionnal test is needed for virt.pool_defined since this is already tested with virt.pool_running. * Add virt.update test parameter In order to allow running dry-runs of virt.update module add a test parameter. This will later be used by the virt states. * Extract virt.defined state from virt.running In order to ensure a virtual guest is defined independently of its status, extract the corresponding code from the virt.running state. This commit also handles the __opts__['test'] for the running state. Since the update call only performs changes if needed, deprecate the update parameter. * Extract virt.network_defined from virt.network_running Just like domains and storage pools, users may want to ensure a network is defined without influencing it's status. Extract the code from network_running state defining the network into a network_defined state. While at it, support __opt__['test'] == True in these states. Updating the network definition in the pool_defined state will come in a future PR. * Fix virt.update to handle None mem and cpu virt.running state now may call virt.update with None mem and cpu parameters. This was not handled in _gen_xml(). Also add some more tests cases matching this for virt.update. * Add support to virt for libvirt loader * Add bhyve compatibility to virt * Fix test_get_hypervisor: mock bhyve * Add virt.all_capabilities In order to get all possible capabilities from a host, the user has to call virt.capabilities, and then loop over the guests and domains before calling virt.domain_capabilities for each of them. This commit embeds all this logic to get them all in a single virt.all_capabilities call. * Virt init disks support (PR#56666) * Fix pylint warning in test_virt.py after blackening * Add pool parameter to virt.define_vol_xml_str * Remove useless default values for disks and vm_name in _disk_profile * virt._gen_vol_xml: move all esx-specifics outside In the near future gen_vol_xml will be able to handle many volume types, not only for ESX volumes. For this, clean up the function from all the ESX-specifics code and move them to the caller code. The volume key and target path values have also been removed since those are read-only elements that should not be provided for volume creation as per https://libvirt.org/formatstorage.html#StorageVol * virt: add more properties to generate volume XML In order to generate almost all the volumes that libvirt can handle, add the type, backingStore, permissions and allocation parameters to the virt._gen_vol_xml() function. Also make the format parameter optional since libvirt has default values depending on the storage backend. * virt.define_vol_xml_str variant using an existing libvirt connection In order to avoid connection multiple times when reusing this function in the virt module, create _define_vol_xml_str not caring about the connection opening and closing. * Add virt.volume_define function In the same vein than pool_define and network_define, expose a volume_define function to let users create a volume without needing to know all of libvirt's XML details. * Add virt.volume_upload function When using volumes the user can just copy the template disk image into the target path. In such a case, the image needs to be uploaded into the volume. * virt.capabilities refactoring Extract the libvirt-handling code from virt.capabilities into a virt._capabilities function accepting an opened libvirt connection. This allows reusing the code in other functions with easy connection handling. * Extract virt.pool_capabilities logic for use with a libvirt connection Te virt.pool_capabilities function computes a lot of interesting values on the virtual storage pool types. Extract the logic of it into virt._pool_capabilities for reuse. * Share libvirt connection in virt.init Since the next commits will introduce more uses of the libvirt connection in virt.init(), start sharing it now. * Add disk volumes support in virt.init In order to support creating VMs with disk on more storage pools like iSCSI, disks, LVM, RBD, etc, virt.init needs to handle disks not only as files, but also as libvirt volumes. * fix libvirtError use libvirtError is not defined, libvirt.libvirtError should be used instead. * virt: let libvirt generate MAC addresses There is no need to generate MAC addresses in the virt module if the user hasn't provided any. This only makes it harder to make the difference between a real mac address change from the user and a new generated one. Now the mac address is not written in the domain XML definition if not provided by the user. This avoids unnecessary changes when applying virt.running. * virt.update handle disk volumes * virt.volume_infos: output backing store as well Since it could be useful to know whether a volume has a backing store, output the path and format of the backing store if any is defined. * virt.volume_infos: output disk format Since the format of a volume may be of interest and could help to tell if two volumes are similar, output this information in the virt.volume_infos function. * Add virt.volume_defined state In order to help creating storage volumes in virtual storage pools from states, add a virt.volume_defined state. * Add volume support to virt._get_disks If a virtual machine has disks of volume types, they will now be reported by the virt._get_disk() function and all the user-exposed functions using it like virt.get_disks(), virt.vm_info() and virt.full_info(). * Add volume disks support to virt.purge() virt.purge will now remove not only the file disks, but also the disk volumes. * Handle RBD volumes as network disks in VM definition libvirt doesn't support attaching RBD storage pool volumes to a VM. For instance, a disk with such a source is invalid: <source pool='rbd-pool' volume='rbd-volume'/> And needs to be replaced with: <source protocol="rbd" name="rbd-pool-name/rbd-volume"> <host name="hostname" port="7000"/> <auth username='myuser'> <secret type='ceph' usage='mypassid'/> </auth> </source> This means that we need to fetch the connection data from the pool definition and convert the volume disk definition into a network one when creating the VM. This also means that when purging the VM we need to search for the pool by looking in every pool's XML to delete the volume. * virt: fix VM creating with disk volume Since volumes in a virtual storage pool of type 'disk' are partitions, then need to be named after the disk name with sequential numbers rather than using the VM and disk names. Also, the format passed by the user is the one used when creating the volume. However in the VM definition for logical and disk volumes, the format should be set to raw. * virt: handle cdrom remote images Libvirt allows to use network images for cdroms. Use them if the image is a remote URL for a cdrom device. * virt.update properly handle removable devices Live attaching / detaching removable devices results in failures. To change the source of those, we need to call updateDeviceFlags instead. * improve boot parameter documentation (PR#57086) * Remove buggy start parameter from virt.pool_running docstring As mentioned by issue #57275, the start parameter in virt.pool_running documentation is not implemented at all. Remove it from the doc. * virt: fix removable_changes definition place * virt: show the proper pool name in get_disks From the user point of view the internal RBD pool name doesn't make sense when listing the disks of a VM. We need then to resolve it to the libvirt pool name if possible. Move the corresponding code from purge to get_disks. * virt: fix updating VM with backing store disks libvirt adds the index attribute in the XML definition when there is a backing store file for a disk. We need to elude it before comparing the sources to avoid trying to recreate disks in such cases. * virt: default cdrom model to ide cdrom devices can't use virtio. Set the default bus to ide for cdroms. * virt.get_disk: output the full URL for cdroms with remote image virt._gen_xml converts the cdroms with remote URL into a network device, but to be coherent with the user input the virt.get_disk() function needs to reassemble the URL in thoses cases. * virt.pool_delete: remove also secret Some pool type require a secret for the authentication on the remote source. Remove the secrets that were added by pool_defined but leave the others in place. * virt.init: cdrom format default to raw cdrom sources can't be of format qcow2. Force raw as the default if needed when creating VM with source_file set for the cdrom. * virt.init: fix disk target names Fixes issue #57477. * virt.update: handle changing cdrom images for devices with remote source When a DVD device on a VM has a remote source, virt.update needs to be able to handle detaching it and attaching a file image live. * virt.init: fix the name of volumes reused in disk-types pools Only compute the partition name if no source_file was provided by the user for a pool of disk type. Co-authored-by: Blacken Salt <jenkins@saltstack.com> Co-authored-by: Firefly <leevn2011@hotmail.com> Co-authored-by: Jeroen Schutrup <jeroenschutrup@hotmail.nl> Co-authored-by: ch3ll <megan.wilhite@gmail.com> Co-authored-by: Blacken Salt <jenkins@saltstack.com> Co-authored-by: Firefly <leevn2011@hotmail.com> Co-authored-by: Jeroen Schutrup <jeroenschutrup@hotmail.nl> Co-authored-by: ch3ll <megan.wilhite@gmail.com> --- changelog/57005.added | 1 + changelog/57275.fixed | 1 + changelog/57477.fixed | 1 + changelog/57497.fixed | 1 + salt/modules/virt.py | 1080 ++++++------------- salt/states/virt.py | 201 +--- salt/templates/virt/libvirt_domain.jinja | 16 +- tests/unit/modules/test_virt.py | 1217 ++++++++++++++++------ tests/unit/states/test_virt.py | 755 +------------- 9 files changed, 1265 insertions(+), 2008 deletions(-) create mode 100644 changelog/57005.added create mode 100644 changelog/57275.fixed create mode 100644 changelog/57477.fixed create mode 100644 changelog/57497.fixed diff --git a/changelog/57005.added b/changelog/57005.added new file mode 100644 index 0000000000..6704541509 --- /dev/null +++ b/changelog/57005.added @@ -0,0 +1 @@ +Add support for disks volumes in virt.running state diff --git a/changelog/57275.fixed b/changelog/57275.fixed new file mode 100644 index 0000000000..6efbe48315 --- /dev/null +++ b/changelog/57275.fixed @@ -0,0 +1 @@ +Remove buggy start parameter from virt.pool_running docstring diff --git a/changelog/57477.fixed b/changelog/57477.fixed new file mode 100644 index 0000000000..f32f32fdfc --- /dev/null +++ b/changelog/57477.fixed @@ -0,0 +1 @@ +virt.init fix the disk target names diff --git a/changelog/57497.fixed b/changelog/57497.fixed new file mode 100644 index 0000000000..24697e108d --- /dev/null +++ b/changelog/57497.fixed @@ -0,0 +1 @@ +Fix volume name for disk-typed pools in virt.defined diff --git a/salt/modules/virt.py b/salt/modules/virt.py index 7314bf1d6e..fb27397baa 100644 --- a/salt/modules/virt.py +++ b/salt/modules/virt.py @@ -71,50 +71,6 @@ The calls not using the libvirt connection setup are: - `libvirt URI format <http://libvirt.org/uri.html#URI_config>`_ - `libvirt authentication configuration <http://libvirt.org/auth.html#Auth_client_config>`_ -Units -========== -.. _virt-units: -.. rubric:: Units specification -.. versionadded:: 3002 - -The string should contain a number optionally followed -by a unit. The number may have a decimal fraction. If -the unit is not given then MiB are set by default. -Units can optionally be given in IEC style (such as MiB), -although the standard single letter style (such as M) is -more convenient. - -Valid units include: - -========== ===== ========== ========== ====== -Standard IEC Standard IEC - Unit Unit Name Name Factor -========== ===== ========== ========== ====== - B Bytes 1 - K KiB Kilobytes Kibibytes 2**10 - M MiB Megabytes Mebibytes 2**20 - G GiB Gigabytes Gibibytes 2**30 - T TiB Terabytes Tebibytes 2**40 - P PiB Petabytes Pebibytes 2**50 - E EiB Exabytes Exbibytes 2**60 - Z ZiB Zettabytes Zebibytes 2**70 - Y YiB Yottabytes Yobibytes 2**80 -========== ===== ========== ========== ====== - -Additional decimal based units: - -====== ======= -Unit Factor -====== ======= -KB 10**3 -MB 10**6 -GB 10**9 -TB 10**12 -PB 10**15 -EB 10**18 -ZB 10**21 -YB 10**24 -====== ======= """ # Special Thanks to Michael Dehann, many of the concepts, and a few structures # of his in the virt func module have been used @@ -136,19 +92,22 @@ from xml.etree import ElementTree from xml.sax import saxutils import jinja2.exceptions -import salt.utils.data import salt.utils.files import salt.utils.json +import salt.utils.network import salt.utils.path import salt.utils.stringutils import salt.utils.templates -import salt.utils.virt +import salt.utils.validate.net +import salt.utils.versions import salt.utils.xmlutil as xmlutil import salt.utils.yaml from salt._compat import ipaddress from salt.exceptions import CommandExecutionError, SaltInvocationError +from salt.ext import six from salt.ext.six.moves import range # pylint: disable=import-error,redefined-builtin from salt.ext.six.moves.urllib.parse import urlparse, urlunparse +from salt.utils.virt import check_remote, download_remote try: import libvirt # pylint: disable=import-error @@ -499,8 +458,6 @@ def _get_disks(conn, dom): """ disks = {} doc = ElementTree.fromstring(dom.XMLDesc(0)) - # Get the path, pool, volume name of each volume we can - all_volumes = _get_all_volumes_paths(conn) for elem in doc.findall("devices/disk"): source = elem.find("source") if source is None: @@ -513,61 +470,13 @@ def _get_disks(conn, dom): extra_properties = None if "dev" in target.attrib: disk_type = elem.get("type") - - def _get_disk_volume_data(pool_name, volume_name): - qemu_target = "{}/{}".format(pool_name, volume_name) - pool = conn.storagePoolLookupByName(pool_name) - vol = pool.storageVolLookupByName(volume_name) - vol_info = vol.info() - extra_properties = { - "virtual size": vol_info[1], - "disk size": vol_info[2], - } - - backing_files = [ - { - "file": node.find("source").get("file"), - "file format": node.find("format").get("type"), - } - for node in elem.findall(".//backingStore[source]") - ] - - if backing_files: - # We had the backing files in a flat list, nest them again. - extra_properties["backing file"] = backing_files[0] - parent = extra_properties["backing file"] - for sub_backing_file in backing_files[1:]: - parent["backing file"] = sub_backing_file - parent = sub_backing_file - - else: - # In some cases the backing chain is not displayed by the domain definition - # Try to see if we have some of it in the volume definition. - vol_desc = ElementTree.fromstring(vol.XMLDesc()) - backing_path = vol_desc.find("./backingStore/path") - backing_format = vol_desc.find("./backingStore/format") - if backing_path is not None: - extra_properties["backing file"] = {"file": backing_path.text} - if backing_format is not None: - extra_properties["backing file"][ - "file format" - ] = backing_format.get("type") - return (qemu_target, extra_properties) - if disk_type == "file": qemu_target = source.get("file", "") if qemu_target.startswith("/dev/zvol/"): disks[target.get("dev")] = {"file": qemu_target, "zfs": True} continue - - if qemu_target in all_volumes.keys(): - # If the qemu_target is a known path, output a volume - volume = all_volumes[qemu_target] - qemu_target, extra_properties = _get_disk_volume_data( - volume["pool"], volume["name"] - ) - elif elem.get("device", "disk") != "cdrom": - # Extract disk sizes, snapshots, backing files + # Extract disk sizes, snapshots, backing files + if elem.get("device", "disk") != "cdrom": try: stdout = subprocess.Popen( [ @@ -589,12 +498,6 @@ def _get_disks(conn, dom): disk.update({"file": "Does not exist"}) elif disk_type == "block": qemu_target = source.get("dev", "") - # If the qemu_target is a known path, output a volume - if qemu_target in all_volumes.keys(): - volume = all_volumes[qemu_target] - qemu_target, extra_properties = _get_disk_volume_data( - volume["pool"], volume["name"] - ) elif disk_type == "network": qemu_target = source.get("protocol") source_name = source.get("name") @@ -633,9 +536,43 @@ def _get_disks(conn, dom): elif disk_type == "volume": pool_name = source.get("pool") volume_name = source.get("volume") - qemu_target, extra_properties = _get_disk_volume_data( - pool_name, volume_name - ) + qemu_target = "{}/{}".format(pool_name, volume_name) + pool = conn.storagePoolLookupByName(pool_name) + vol = pool.storageVolLookupByName(volume_name) + vol_info = vol.info() + extra_properties = { + "virtual size": vol_info[1], + "disk size": vol_info[2], + } + + backing_files = [ + { + "file": node.find("source").get("file"), + "file format": node.find("format").get("type"), + } + for node in elem.findall(".//backingStore[source]") + ] + + if backing_files: + # We had the backing files in a flat list, nest them again. + extra_properties["backing file"] = backing_files[0] + parent = extra_properties["backing file"] + for sub_backing_file in backing_files[1:]: + parent["backing file"] = sub_backing_file + parent = sub_backing_file + + else: + # In some cases the backing chain is not displayed by the domain definition + # Try to see if we have some of it in the volume definition. + vol_desc = ElementTree.fromstring(vol.XMLDesc()) + backing_path = vol_desc.find("./backingStore/path") + backing_format = vol_desc.find("./backingStore/format") + if backing_path is not None: + extra_properties["backing file"] = {"file": backing_path.text} + if backing_format is not None: + extra_properties["backing file"][ + "file format" + ] = backing_format.get("type") if not qemu_target: continue @@ -676,258 +613,26 @@ def _libvirt_creds(): return {"user": user, "group": group} -def _migrate(dom, dst_uri, **kwargs): +def _get_migrate_command(): """ - Migrate the domain object from its current host to the destination - host given by URI. - - :param dom: domain object to migrate - :param dst_uri: destination URI - :param kwargs: - - live: Use live migration. Default value is True. - - persistent: Leave the domain persistent on destination host. - Default value is True. - - undefinesource: Undefine the domain on the source host. - Default value is True. - - offline: If set to True it will migrate the domain definition - without starting the domain on destination and without - stopping it on source host. Default value is False. - - max_bandwidth: The maximum bandwidth (in MiB/s) that will be used. - - max_downtime: Set maximum tolerable downtime for live-migration. - The value represents a number of milliseconds the guest - is allowed to be down at the end of live migration. - - parallel_connections: Specify a number of parallel network connections - to be used to send memory pages to the destination host. - - compressed: Activate compression. - - comp_methods: A comma-separated list of compression methods. Supported - methods are "mt" and "xbzrle" and can be used in any - combination. QEMU defaults to "xbzrle". - - comp_mt_level: Set compression level. Values are in range from 0 to 9, - where 1 is maximum speed and 9 is maximum compression. - - comp_mt_threads: Set number of compress threads on source host. - - comp_mt_dthreads: Set number of decompress threads on target host. - - comp_xbzrle_cache: Set the size of page cache for xbzrle compression in bytes. - - copy_storage: Migrate non-shared storage. It must be one of the following - values: all (full disk copy) or incremental (Incremental copy) - - postcopy: Enable the use of post-copy migration. - - postcopy_bandwidth: The maximum bandwidth allowed in post-copy phase. (MiB/s) - - username: Username to connect with target host - - password: Password to connect with target host + Returns the command shared by the different migration types """ - flags = 0 - params = {} - migrated_state = libvirt.VIR_DOMAIN_RUNNING_MIGRATED - - if kwargs.get("live", True): - flags |= libvirt.VIR_MIGRATE_LIVE - - if kwargs.get("persistent", True): - flags |= libvirt.VIR_MIGRATE_PERSIST_DEST - - if kwargs.get("undefinesource", True): - flags |= libvirt.VIR_MIGRATE_UNDEFINE_SOURCE - - max_bandwidth = kwargs.get("max_bandwidth") - if max_bandwidth: - try: - bandwidth_value = int(max_bandwidth) - except ValueError: - raise SaltInvocationError( - "Invalid max_bandwidth value: {}".format(max_bandwidth) - ) - dom.migrateSetMaxSpeed(bandwidth_value) - - max_downtime = kwargs.get("max_downtime") - if max_downtime: - try: - downtime_value = int(max_downtime) - except ValueError: - raise SaltInvocationError( - "Invalid max_downtime value: {}".format(max_downtime) - ) - dom.migrateSetMaxDowntime(downtime_value) - - if kwargs.get("offline") is True: - flags |= libvirt.VIR_MIGRATE_OFFLINE - migrated_state = libvirt.VIR_DOMAIN_RUNNING_UNPAUSED - - if kwargs.get("compressed") is True: - flags |= libvirt.VIR_MIGRATE_COMPRESSED - - comp_methods = kwargs.get("comp_methods") - if comp_methods: - params[libvirt.VIR_MIGRATE_PARAM_COMPRESSION] = comp_methods.split(",") - - comp_options = { - "comp_mt_level": libvirt.VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL, - "comp_mt_threads": libvirt.VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS, - "comp_mt_dthreads": libvirt.VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS, - "comp_xbzrle_cache": libvirt.VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE, - } - - for (comp_option, param_key) in comp_options.items(): - comp_option_value = kwargs.get(comp_option) - if comp_option_value: - try: - params[param_key] = int(comp_option_value) - except ValueError: - raise SaltInvocationError("Invalid {} value".format(comp_option)) - - parallel_connections = kwargs.get("parallel_connections") - if parallel_connections: - try: - params[libvirt.VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS] = int( - parallel_connections - ) - except ValueError: - raise SaltInvocationError("Invalid parallel_connections value") - flags |= libvirt.VIR_MIGRATE_PARALLEL - - if __salt__["config.get"]("virt:tunnel"): - if parallel_connections: - raise SaltInvocationError( - "Parallel migration isn't compatible with tunneled migration" - ) - flags |= libvirt.VIR_MIGRATE_PEER2PEER - flags |= libvirt.VIR_MIGRATE_TUNNELLED - - if kwargs.get("postcopy") is True: - flags |= libvirt.VIR_MIGRATE_POSTCOPY - - postcopy_bandwidth = kwargs.get("postcopy_bandwidth") - if postcopy_bandwidth: - try: - postcopy_bandwidth_value = int(postcopy_bandwidth) - except ValueError: - raise SaltInvocationError("Invalid postcopy_bandwidth value") - dom.migrateSetMaxSpeed( - postcopy_bandwidth_value, - flags=libvirt.VIR_DOMAIN_MIGRATE_MAX_SPEED_POSTCOPY, - ) - - copy_storage = kwargs.get("copy_storage") - if copy_storage: - if copy_storage == "all": - flags |= libvirt.VIR_MIGRATE_NON_SHARED_DISK - elif copy_storage in ["inc", "incremental"]: - flags |= libvirt.VIR_MIGRATE_NON_SHARED_INC - else: - raise SaltInvocationError("invalid copy_storage value") - try: - state = False - dst_conn = __get_conn( - connection=dst_uri, - username=kwargs.get("username"), - password=kwargs.get("password"), + tunnel = __salt__["config.get"]("virt:tunnel") + if tunnel: + return ( + "virsh migrate --p2p --tunnelled --live --persistent " "--undefinesource " ) - new_dom = dom.migrate3(dconn=dst_conn, params=params, flags=flags) - if new_dom: - state = new_dom.state() - dst_conn.close() - return state and migrated_state in state - except libvirt.libvirtError as err: - dst_conn.close() - raise CommandExecutionError(err.get_error_message()) - - -def _get_volume_path(pool, volume_name): - """ - Get the path to a volume. If the volume doesn't exist, compute its path from the pool one. - """ - if volume_name in pool.listVolumes(): - volume = pool.storageVolLookupByName(volume_name) - volume_xml = ElementTree.fromstring(volume.XMLDesc()) - return volume_xml.find("./target/path").text - - # Get the path from the pool if the volume doesn't exist yet - pool_xml = ElementTree.fromstring(pool.XMLDesc()) - pool_path = pool_xml.find("./target/path").text - return pool_path + "/" + volume_name + return "virsh migrate --live --persistent --undefinesource " -def _disk_from_pool(conn, pool, pool_xml, volume_name): +def _get_target(target, ssh): """ - Create a disk definition out of the pool XML and volume name. - The aim of this function is to replace the volume-based definition when not handled by libvirt. - It returns the disk Jinja context to be used when creating the VM + Compute libvirt URL for target migration host. """ - pool_type = pool_xml.get("type") - disk_context = {} - - # handle dir, fs and netfs - if pool_type in ["dir", "netfs", "fs"]: - disk_context["type"] = "file" - disk_context["source_file"] = _get_volume_path(pool, volume_name) - - elif pool_type in ["logical", "disk", "iscsi", "scsi"]: - disk_context["type"] = "block" - disk_context["format"] = "raw" - disk_context["source_file"] = _get_volume_path(pool, volume_name) - - elif pool_type in ["rbd", "gluster", "sheepdog"]: - # libvirt can't handle rbd, gluster and sheepdog as volumes - disk_context["type"] = "network" - disk_context["protocol"] = pool_type - # Copy the hosts from the pool definition - disk_context["hosts"] = [ - {"name": host.get("name"), "port": host.get("port")} - for host in pool_xml.findall(".//host") - ] - dir_node = pool_xml.find("./source/dir") - # Gluster and RBD need pool/volume name - name_node = pool_xml.find("./source/name") - if name_node is not None: - disk_context["volume"] = "{}/{}".format(name_node.text, volume_name) - # Copy the authentication if any for RBD - auth_node = pool_xml.find("./source/auth") - if auth_node is not None: - username = auth_node.get("username") - secret_node = auth_node.find("./secret") - usage = secret_node.get("usage") - if not usage: - # Get the usage from the UUID - uuid = secret_node.get("uuid") - usage = conn.secretLookupByUUIDString(uuid).usageID() - disk_context["auth"] = { - "type": "ceph", - "username": username, - "usage": usage, - } - - return disk_context - - -def _handle_unit(s, def_unit="m"): - """ - Handle the unit conversion, return the value in bytes - """ - m = re.match(r"(?P<value>[0-9.]*)\s*(?P<unit>.*)$", str(s).strip()) - value = m.group("value") - # default unit - unit = m.group("unit").lower() or def_unit - try: - value = int(value) - except ValueError: - try: - value = float(value) - except ValueError: - raise SaltInvocationError("invalid number") - # flag for base ten - dec = False - if re.match(r"[kmgtpezy]b$", unit): - dec = True - elif not re.match(r"(b|[kmgtpezy](ib)?)$", unit): - raise SaltInvocationError("invalid units") - p = "bkmgtpezy".index(unit[0]) - value *= 10 ** (p * 3) if dec else 2 ** (p * 10) - return int(value) - - -def nesthash(): - """ - create default dict that allows arbitrary level of nesting - """ - return collections.defaultdict(nesthash) + proto = "qemu" + if ssh: + proto += "+ssh" + return " {}://{}/{}".format(proto, target, "system") def _gen_xml( @@ -942,31 +647,18 @@ def _gen_xml( arch, graphics=None, boot=None, - boot_dev=None, **kwargs ): """ Generate the XML string to define a libvirt VM """ + mem = int(mem) * 1024 # MB context = { "hypervisor": hypervisor, "name": name, "cpu": str(cpu), + "mem": str(mem), } - - context["mem"] = nesthash() - if isinstance(mem, int): - mem = int(mem) * 1024 # MB - context["mem"]["boot"] = str(mem) - context["mem"]["current"] = str(mem) - elif isinstance(mem, dict): - for tag, val in mem.items(): - if val: - if tag == "slots": - context["mem"]["slots"] = "{}='{}'".format(tag, val) - else: - context["mem"][tag] = str(int(_handle_unit(val) / 1024)) - if hypervisor in ["qemu", "kvm"]: context["controller_model"] = False elif hypervisor == "vmware": @@ -988,17 +680,15 @@ def _gen_xml( graphics = None context["graphics"] = graphics - context["boot_dev"] = boot_dev.split() if boot_dev is not None else ["hd"] + if "boot_dev" in kwargs: + context["boot_dev"] = [] + for dev in kwargs["boot_dev"].split(): + context["boot_dev"].append(dev) + else: + context["boot_dev"] = ["hd"] context["boot"] = boot if boot else {} - # if efi parameter is specified, prepare os_attrib - efi_value = context["boot"].get("efi", None) if boot else None - if efi_value is True: - context["boot"]["os_attrib"] = "firmware='efi'" - elif efi_value is not None and type(efi_value) != bool: - raise SaltInvocationError("Invalid efi value") - if os_type == "xen": # Compute the Xen PV boot method if __grains__["os_family"] == "Suse": @@ -1047,16 +737,41 @@ def _gen_xml( elif disk.get("pool"): disk_context["volume"] = disk["filename"] # If we had no source_file, then we want a volume - pool = conn.storagePoolLookupByName(disk["pool"]) - pool_xml = ElementTree.fromstring(pool.XMLDesc()) + pool_xml = ElementTree.fromstring( + conn.storagePoolLookupByName(disk["pool"]).XMLDesc() + ) pool_type = pool_xml.get("type") - - # For Xen VMs convert all pool types (issue #58333) - if hypervisor == "xen" or pool_type in ["rbd", "gluster", "sheepdog"]: - disk_context.update( - _disk_from_pool(conn, pool, pool_xml, disk_context["volume"]) - ) - + if pool_type in ["rbd", "gluster", "sheepdog"]: + # libvirt can't handle rbd, gluster and sheepdog as volumes + disk_context["type"] = "network" + disk_context["protocol"] = pool_type + # Copy the hosts from the pool definition + disk_context["hosts"] = [ + {"name": host.get("name"), "port": host.get("port")} + for host in pool_xml.findall(".//host") + ] + dir_node = pool_xml.find("./source/dir") + # Gluster and RBD need pool/volume name + name_node = pool_xml.find("./source/name") + if name_node is not None: + disk_context["volume"] = "{}/{}".format( + name_node.text, disk_context["volume"] + ) + # Copy the authentication if any for RBD + auth_node = pool_xml.find("./source/auth") + if auth_node is not None: + username = auth_node.get("username") + secret_node = auth_node.find("./secret") + usage = secret_node.get("usage") + if not usage: + # Get the usage from the UUID + uuid = secret_node.get("uuid") + usage = conn.secretLookupByUUIDString(uuid).usageID() + disk_context["auth"] = { + "type": "ceph", + "username": username, + "usage": usage, + } else: if pool_type in ["disk", "logical"]: # The volume format for these types doesn't match the driver format in the VM @@ -1086,6 +801,7 @@ def _gen_xml( except jinja2.exceptions.TemplateNotFound: log.error("Could not load template %s", fn_) return "" + return template.render(**context) @@ -1521,7 +1237,7 @@ def _disk_profile(conn, profile, hypervisor, disks, vm_name): overlay = {"device": "disk", "model": "virtio"} elif hypervisor == "xen": overlay = {"device": "disk", "model": "xen"} - elif hypervisor == "bhyve": + elif hypervisor in ["bhyve"]: overlay = {"format": "raw", "model": "virtio", "sparse_volume": False} else: overlay = {} @@ -1566,7 +1282,7 @@ def _disk_profile(conn, profile, hypervisor, disks, vm_name): disk["format"] = ( "qcow2" if disk.get("device", "disk") != "cdrom" else "raw" ) - elif vm_name and disk.get("device", "disk") == "disk": + elif disk.get("device", "disk") == "disk": _fill_disk_filename(conn, vm_name, disk, hypervisor, pool_caps) return disklist @@ -1803,68 +1519,30 @@ def _handle_remote_boot_params(orig_boot): new_boot = orig_boot.copy() keys = orig_boot.keys() cases = [ - {"efi"}, - {"kernel", "initrd", "efi"}, - {"kernel", "initrd", "cmdline", "efi"}, {"loader", "nvram"}, {"kernel", "initrd"}, {"kernel", "initrd", "cmdline"}, - {"kernel", "initrd", "loader", "nvram"}, - {"kernel", "initrd", "cmdline", "loader", "nvram"}, + {"loader", "nvram", "kernel", "initrd"}, + {"loader", "nvram", "kernel", "initrd", "cmdline"}, ] try: if keys in cases: for key in keys: - if key == "efi" and type(orig_boot.get(key)) == bool: - new_boot[key] = orig_boot.get(key) - elif orig_boot.get(key) is not None and salt.utils.virt.check_remote( - orig_boot.get(key) - ): + if orig_boot.get(key) is not None and check_remote(orig_boot.get(key)): if saltinst_dir is None: os.makedirs(CACHE_DIR) saltinst_dir = CACHE_DIR - new_boot[key] = salt.utils.virt.download_remote( - orig_boot.get(key), saltinst_dir - ) + new_boot[key] = download_remote(orig_boot.get(key), saltinst_dir) return new_boot else: raise SaltInvocationError( - "Invalid boot parameters,It has to follow this combination: [(kernel, initrd) or/and cmdline] or/and [(loader, nvram) or efi]" + "Invalid boot parameters, (kernel, initrd) or/and (loader, nvram) must be both present" ) except Exception as err: # pylint: disable=broad-except raise err -def _handle_efi_param(boot, desc): - """ - Checks if boot parameter contains efi boolean value, if so, handles the firmware attribute. - :param boot: The boot parameters passed to the init or update functions. - :param desc: The XML description of that domain. - :return: A boolean value. - """ - efi_value = boot.get("efi", None) if boot else None - parent_tag = desc.find("os") - os_attrib = parent_tag.attrib - - # newly defined vm without running, loader tag might not be filled yet - if efi_value is False and os_attrib != {}: - parent_tag.attrib.pop("firmware", None) - return True - - # check the case that loader tag might be present. This happens after the vm ran - elif type(efi_value) == bool and os_attrib == {}: - if efi_value is True and parent_tag.find("loader") is None: - parent_tag.set("firmware", "efi") - if efi_value is False and parent_tag.find("loader") is not None: - parent_tag.remove(parent_tag.find("loader")) - parent_tag.remove(parent_tag.find("nvram")) - return True - elif type(efi_value) != bool: - raise SaltInvocationError("Invalid efi value") - return False - - def init( name, cpu, @@ -1885,7 +1563,6 @@ def init( os_type=None, arch=None, boot=None, - boot_dev=None, **kwargs ): """ @@ -1893,28 +1570,7 @@ def init( :param name: name of the virtual machine to create :param cpu: Number of virtual CPUs to assign to the virtual machine - :param mem: Amount of memory to allocate to the virtual machine in MiB. Since 3002, a dictionary can be used to - contain detailed configuration which support memory allocation or tuning. Supported parameters are ``boot``, - ``current``, ``max``, ``slots``, ``hard_limit``, ``soft_limit``, ``swap_hard_limit`` and ``min_guarantee``. The - structure of the dictionary is documented in :ref:`init-mem-def`. Both decimal and binary base are supported. - Detail unit specification is documented in :ref:`virt-units`. Please note that the value for ``slots`` must be - an integer. - - .. code-block:: python - - { - 'boot': 1g, - 'current': 1g, - 'max': 1g, - 'slots': 10, - 'hard_limit': '1024' - 'soft_limit': '512m' - 'swap_hard_limit': '1g' - 'min_guarantee': '512mib' - } - - .. versionchanged:: 3002 - + :param mem: Amount of memory to allocate to the virtual machine in MiB. :param nic: NIC profile to use (Default: ``'default'``). The profile interfaces can be customized / extended with the interfaces parameter. If set to ``None``, no profile will be used. @@ -1976,8 +1632,7 @@ def init( This is an optional parameter, all of the keys are optional within the dictionary. The structure of the dictionary is documented in :ref:`init-boot-def`. If a remote path is provided to kernel or initrd, salt will handle the downloading of the specified remote file and modify the XML accordingly. - To boot VM with UEFI, specify loader and nvram path or specify 'efi': ``True`` if your libvirtd version - is >= 5.2.0 and QEMU >= 3.0.0. + To boot VM with UEFI, specify loader and nvram path. .. versionadded:: 3000 @@ -1991,12 +1646,6 @@ def init( 'nvram': '/usr/share/OVMF/OVMF_VARS.ms.fd' } - :param boot_dev: - Space separated list of devices to boot from sorted by decreasing priority. - Values can be ``hd``, ``fd``, ``cdrom`` or ``network``. - - By default, the value will ``"hd"``. - .. _init-boot-def: .. rubric:: Boot parameters definition @@ -2015,47 +1664,12 @@ def init( loader The path to the UEFI binary loader to use. - .. versionadded:: 3001 + .. versionadded:: sodium nvram The path to the UEFI data template. The file will be copied when creating the virtual machine. - .. versionadded:: 3001 - - efi - A boolean value. - - .. versionadded:: sodium - - .. _init-mem-def: - - .. rubric:: Memory parameter definition - - Memory parameter can contain the following properties: - - boot - The maximum allocation of memory for the guest at boot time - - current - The actual allocation of memory for the guest - - max - The run time maximum memory allocation of the guest - - slots - specifies the number of slots available for adding memory to the guest - - hard_limit - the maximum memory the guest can use - - soft_limit - memory limit to enforce during memory contention - - swap_hard_limit - the maximum memory plus swap the guest can use - - min_guarantee - the guaranteed minimum memory allocation for the guest + .. versionadded:: sodium .. _init-nic-def: @@ -2099,7 +1713,7 @@ def init( Path to the folder or name of the pool where disks should be created. (Default: depends on hypervisor and the virt:storagepool configuration) - .. versionchanged:: 3001 + .. versionchanged:: sodium If the value contains no '/', it is considered a pool name where to create a volume. Using volumes will be mandatory for some pools types like rdb, iscsi, etc. @@ -2120,7 +1734,7 @@ def init( ``True`` to create a QCOW2 disk image with ``image`` as backing file. If ``False`` the file pointed to by the ``image`` property will simply be copied. (Default: ``False``) - .. versionchanged:: 3001 + .. versionchanged:: sodium This property is only valid on path-based disks, not on volumes. To create a volume with a backing store, set the ``backing_store_path`` and ``backing_store_format`` properties. @@ -2129,20 +1743,20 @@ def init( Path to the backing store image to use. This can also be the name of a volume to use as backing store within the same pool. - .. versionadded:: 3001 + .. versionadded:: sodium backing_store_format Image format of the disk or volume to use as backing store. This property is mandatory when using ``backing_store_path`` to avoid `problems <https://libvirt.org/kbase/backing_chains.html#troubleshooting>`_ - .. versionadded:: 3001 + .. versionadded:: sodium source_file Absolute path to the disk image to use. Not to be confused with ``image`` parameter. This parameter is useful to use disk images that are created outside of this module. Can also be ``None`` for devices that have no associated image like cdroms. - .. versionchanged:: 3001 + .. versionchanged:: sodium For volume disks, this can be the name of a volume already existing in the storage pool. @@ -2241,8 +1855,6 @@ def init( for x in y } ) - if len(hypervisors) == 0: - raise SaltInvocationError("No supported hypervisors were found") virt_hypervisor = "kvm" if "kvm" in hypervisors else hypervisors[0] # esxi used to be a possible value for the hypervisor: map it to vmware since it's the same @@ -2350,10 +1962,8 @@ def init( arch, graphics, boot, - boot_dev, **kwargs ) - log.debug("New virtual machine definition: %s", vm_xml) conn.defineXML(vm_xml) except libvirt.libvirtError as err: conn.close() @@ -2517,11 +2127,11 @@ def _get_disk_target(targets, disks_count, prefix): :param disks: the number of disks :param prefix: the prefix of the target name, i.e. "hd" """ - for i in range(disks_count): - ret = "{}{}".format(prefix, string.ascii_lowercase[i]) - if ret not in targets: - return ret - return None + return [ + "{}{}".format(prefix, string.ascii_lowercase[i]) + for i in range(disks_count) + if "{}{}".format(prefix, string.ascii_lowercase[i]) not in targets + ][0] def _diff_disk_lists(old, new): @@ -2638,17 +2248,8 @@ def update( Refer to :ref:`init-boot-def` for the complete boot parameter description. - To update any boot parameters, specify the new path for each. To remove any boot parameters, pass ``None`` object, - for instance: 'kernel': ``None``. To switch back to BIOS boot, specify ('loader': ``None`` and 'nvram': ``None``) - or 'efi': ``False``. Please note that ``None`` is mapped to ``null`` in sls file, pass ``null`` in sls file instead. - - SLS file Example: - - .. code-block:: yaml - - - boot: - loader: null - nvram: null + To update any boot parameters, specify the new path for each. To remove any boot parameters, + pass a None object, for instance: 'kernel': ``None``. .. versionadded:: 3000 @@ -2704,6 +2305,7 @@ def update( new_desc = ElementTree.fromstring( _gen_xml( + conn, name, cpu or 0, mem or 0, @@ -2725,121 +2327,67 @@ def update( cpu_node.set("current", str(cpu)) need_update = True - def _set_loader(node, value): - salt.utils.xmlutil.set_node_text(node, value) - if value is not None: - node.set("readonly", "yes") - node.set("type", "pflash") + # Update the kernel boot parameters + boot_tags = ["kernel", "initrd", "cmdline", "loader", "nvram"] + parent_tag = desc.find("os") - def _set_nvram(node, value): - node.set("template", value) + # We need to search for each possible subelement, and update it. + for tag in boot_tags: + # The Existing Tag... + found_tag = parent_tag.find(tag) - def _set_with_byte_unit(node, value): - node.text = str(value) - node.set("unit", "bytes") + # The new value + boot_tag_value = boot.get(tag, None) if boot else None - def _get_with_unit(node): - unit = node.get("unit", "KiB") - # _handle_unit treats bytes as invalid unit for the purpose of consistency - unit = unit if unit != "bytes" else "b" - value = node.get("memory") or node.text - return _handle_unit("{}{}".format(value, unit)) if value else None + # Existing tag is found and values don't match + if found_tag is not None and found_tag.text != boot_tag_value: - old_mem = int(_get_with_unit(desc.find("memory")) / 1024) + # If the existing tag is found, but the new value is None + # remove it. If the existing tag is found, and the new value + # doesn't match update it. In either case, mark for update. + if boot_tag_value is None and boot is not None and parent_tag is not None: + parent_tag.remove(found_tag) + else: + found_tag.text = boot_tag_value - # Update the kernel boot parameters - params_mapping = [ - {"path": "boot:kernel", "xpath": "os/kernel"}, - {"path": "boot:initrd", "xpath": "os/initrd"}, - {"path": "boot:cmdline", "xpath": "os/cmdline"}, - {"path": "boot:loader", "xpath": "os/loader", "set": _set_loader}, - {"path": "boot:nvram", "xpath": "os/nvram", "set": _set_nvram}, - # Update the memory, note that libvirt outputs all memory sizes in KiB - { - "path": "mem", - "xpath": "memory", - "convert": _handle_unit, - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem", - "xpath": "currentMemory", - "convert": _handle_unit, - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:max", - "convert": _handle_unit, - "xpath": "maxMemory", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:boot", - "convert": _handle_unit, - "xpath": "memory", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:current", - "convert": _handle_unit, - "xpath": "currentMemory", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:slots", - "xpath": "maxMemory", - "get": lambda n: n.get("slots"), - "set": lambda n, v: n.set("slots", str(v)), - "del": salt.utils.xmlutil.del_attribute("slots", ["unit"]), - }, - { - "path": "mem:hard_limit", - "convert": _handle_unit, - "xpath": "memtune/hard_limit", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:soft_limit", - "convert": _handle_unit, - "xpath": "memtune/soft_limit", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:swap_hard_limit", - "convert": _handle_unit, - "xpath": "memtune/swap_hard_limit", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "mem:min_guarantee", - "convert": _handle_unit, - "xpath": "memtune/min_guarantee", - "get": _get_with_unit, - "set": _set_with_byte_unit, - }, - { - "path": "boot_dev:{dev}", - "xpath": "os/boot[$dev]", - "get": lambda n: n.get("dev"), - "set": lambda n, v: n.set("dev", v), - "del": salt.utils.xmlutil.del_attribute("dev"), - }, - ] + # If the existing tag is loader or nvram, we need to update the corresponding attribute + if found_tag.tag == "loader" and boot_tag_value is not None: + found_tag.set("readonly", "yes") + found_tag.set("type", "pflash") - data = {k: v for k, v in locals().items() if bool(v)} - if boot_dev: - data["boot_dev"] = {i + 1: dev for i, dev in enumerate(boot_dev.split())} - need_update = ( - salt.utils.xmlutil.change_xml(desc, data, params_mapping) or need_update - ) + if found_tag.tag == "nvram" and boot_tag_value is not None: + found_tag.set("template", found_tag.text) + found_tag.text = None + + need_update = True + + # Need to check for parent tag, and add it if it does not exist. + # Add a subelement and set the value to the new value, and then + # mark for update. + if parent_tag is not None: + child_tag = ElementTree.SubElement(parent_tag, tag) + else: + new_parent_tag = ElementTree.Element("os") + child_tag = ElementTree.SubElement(new_parent_tag, tag) + + child_tag.text = boot_tag_value + + # If the newly created tag is loader or nvram, we need to update the corresponding attribute + if child_tag.tag == "loader": + child_tag.set("readonly", "yes") + child_tag.set("type", "pflash") + + if child_tag.tag == "nvram": + child_tag.set("template", child_tag.text) + child_tag.text = None + + # Update the memory, note that libvirt outputs all memory sizes in KiB + for mem_node_name in ["memory", "currentMemory"]: + mem_node = desc.find(mem_node_name) + if mem and int(mem_node.text) != mem * 1024: + mem_node.text = str(mem) + mem_node.set("unit", "MiB") + need_update = True # Update the XML definition with the new disks and diff changes devices_node = desc.find("devices") @@ -2869,18 +2417,22 @@ def update( # Set the new definition if need_update: # Create missing disks if needed - if changes["disk"]: - for idx, item in enumerate(changes["disk"]["sorted"]): - source_file = all_disks[idx]["source_file"] - if ( - item in changes["disk"]["new"] - and source_file - and not os.path.isfile(source_file) - and not test - ): - _qemu_image_create(all_disks[idx]) - try: + if changes["disk"]: + for idx, item in enumerate(changes["disk"]["sorted"]): + source_file = all_disks[idx].get("source_file") + # We don't want to create image disks for cdrom devices + if all_disks[idx].get("device", "disk") == "cdrom": + continue + if ( + item in changes["disk"]["new"] + and source_file + and not os.path.isfile(source_file) + ): + _qemu_image_create(all_disks[idx]) + elif item in changes["disk"]["new"] and not source_file: + _disk_volume_create(conn, all_disks[idx]) + if not test: conn.defineXML( salt.utils.stringutils.to_str(ElementTree.tostring(desc)) @@ -2905,24 +2457,13 @@ def update( } ) if mem: - if isinstance(mem, dict): - # setMemoryFlags takes memory amount in KiB - new_mem = ( - int(_handle_unit(mem.get("current")) / 1024) - if "current" in mem - else None - ) - elif isinstance(mem, int): - new_mem = int(mem * 1024) - - if old_mem != new_mem and new_mem is not None: - commands.append( - { - "device": "mem", - "cmd": "setMemoryFlags", - "args": [new_mem, libvirt.VIR_DOMAIN_AFFECT_LIVE], - } - ) + commands.append( + { + "device": "mem", + "cmd": "setMemoryFlags", + "args": [mem * 1024, libvirt.VIR_DOMAIN_AFFECT_LIVE], + } + ) # Look for removable device source changes new_disks = [] @@ -3663,12 +3204,13 @@ def get_profiles(hypervisor=None, **kwargs): .. code-block:: bash salt '*' virt.get_profiles - salt '*' virt.get_profiles hypervisor=vmware + salt '*' virt.get_profiles hypervisor=esxi """ + ret = {} + # Use the machine types as possible values # Prefer 'kvm' over the others if available - conn = __get_conn(**kwargs) - caps = _capabilities(conn) + caps = capabilities(**kwargs) hypervisors = sorted( { x @@ -3676,24 +3218,24 @@ def get_profiles(hypervisor=None, **kwargs): for x in y } ) - if len(hypervisors) == 0: - raise SaltInvocationError("No supported hypervisors were found") + default_hypervisor = "kvm" if "kvm" in hypervisors else hypervisors[0] if not hypervisor: - hypervisor = "kvm" if "kvm" in hypervisors else hypervisors[0] - - ret = { - "disk": {"default": _disk_profile(conn, "default", hypervisor, [], None)}, - "nic": {"default": _nic_profile("default", hypervisor)}, - } + hypervisor = default_hypervisor virtconf = __salt__["config.get"]("virt", {}) - - for profile in virtconf.get("disk", []): - ret["disk"][profile] = _disk_profile(conn, profile, hypervisor, [], None) - - for profile in virtconf.get("nic", []): - ret["nic"][profile] = _nic_profile(profile, hypervisor) - + for typ in ["disk", "nic"]: + _func = getattr(sys.modules[__name__], "_{}_profile".format(typ)) + ret[typ] = { + "default": _func( + "default", hypervisor if hypervisor else default_hypervisor + ) + } + if typ in virtconf: + ret.setdefault(typ, {}) + for prf in virtconf[typ]: + ret[typ][prf] = _func( + prf, hypervisor if hypervisor else default_hypervisor + ) return ret @@ -4061,7 +3603,7 @@ def define_vol_xml_str( storage pool name to define the volume in. If defined, this parameter will override the configuration setting. - .. versionadded:: 3001 + .. versionadded:: Sodium :param connection: libvirt connection URI, overriding defaults .. versionadded:: 2019.2.0 @@ -4106,7 +3648,7 @@ def define_vol_xml_path(path, pool=None, **kwargs): storage pool name to define the volume in. If defined, this parameter will override the configuration setting. - .. versionadded:: 3001 + .. versionadded:: Sodium :param connection: libvirt connection URI, overriding defaults .. versionadded:: 2019.2.0 @@ -4133,7 +3675,7 @@ def define_vol_xml_path(path, pool=None, **kwargs): return False -def migrate_non_shared(vm_, target, ssh=False, **kwargs): +def migrate_non_shared(vm_, target, ssh=False): """ Attempt to execute non-shared storage "all" migration @@ -4191,17 +3733,17 @@ def migrate_non_shared(vm_, target, ssh=False, **kwargs): For more details on tunnelled data migrations, report to https://libvirt.org/migration.html#transporttunnel """ - salt.utils.versions.warn_until( - "Silicon", - "The 'migrate_non_shared' feature has been deprecated. " - "Use 'migrate' with copy_storage='all' instead.", + cmd = ( + _get_migrate_command() + " --copy-storage-all " + vm_ + _get_target(target, ssh) ) - return migrate(vm_, target, ssh, copy_storage="all", **kwargs) + + stdout = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).communicate()[0] + return salt.utils.stringutils.to_str(stdout) -def migrate_non_shared_inc(vm_, target, ssh=False, **kwargs): +def migrate_non_shared_inc(vm_, target, ssh=False): """ - Attempt to execute non-shared storage "inc" migration + Attempt to execute non-shared storage "all" migration :param vm_: domain name :param target: target libvirt host name @@ -4257,15 +3799,15 @@ def migrate_non_shared_inc(vm_, target, ssh=False, **kwargs): For more details on tunnelled data migrations, report to https://libvirt.org/migration.html#transporttunnel """ - salt.utils.versions.warn_until( - "Silicon", - "The 'migrate_non_shared_inc' feature has been deprecated. " - "Use 'migrate' with copy_storage='inc' instead.", + cmd = ( + _get_migrate_command() + " --copy-storage-inc " + vm_ + _get_target(target, ssh) ) - return migrate(vm_, target, ssh, copy_storage="inc", **kwargs) + + stdout = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).communicate()[0] + return salt.utils.stringutils.to_str(stdout) -def migrate(vm_, target, ssh=False, **kwargs): +def migrate(vm_, target, ssh=False): """ Shared storage migration @@ -4328,51 +3870,10 @@ def migrate(vm_, target, ssh=False, **kwargs): For more details on tunnelled data migrations, report to https://libvirt.org/migration.html#transporttunnel """ + cmd = _get_migrate_command() + " " + vm_ + _get_target(target, ssh) - if ssh: - salt.utils.versions.warn_until( - "Silicon", - "The 'ssh' argument has been deprecated and " - "will be removed in a future release. " - "Use libvirt URI string 'target' instead.", - ) - - conn = __get_conn() - dom = _get_domain(conn, vm_) - - if not urlparse(target).scheme: - proto = "qemu" - if ssh: - proto += "+ssh" - dst_uri = "{}://{}/system".format(proto, target) - else: - dst_uri = target - - ret = _migrate(dom, dst_uri, **kwargs) - conn.close() - return ret - - -def migrate_start_postcopy(vm_): - """ - Starts post-copy migration. This function has to be called - while live migration is in progress and it has been initiated - with the `postcopy=True` option. - - CLI Example: - - .. code-block:: bash - - salt '*' virt.migrate_start_postcopy <domain> - """ - conn = __get_conn() - dom = _get_domain(conn, vm_) - try: - dom.migrateStartPostCopy() - except libvirt.libvirtError as err: - conn.close() - raise CommandExecutionError(err.get_error_message()) - conn.close() + stdout = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).communicate()[0] + return salt.utils.stringutils.to_str(stdout) def seed_non_shared_migrate(disks, force=False): @@ -5206,14 +4707,15 @@ def _parse_caps_guest(guest): # without possibility to toggle them. # Some guests may also have no feature at all (xen pv for instance) features_nodes = guest.find("features") - if features_nodes is not None and child is not None: + if features_nodes is not None: result["features"] = { child.tag: { - "toggle": child.get("toggle", "no") == "yes", - "default": child.get("default", "on") == "on", + "toggle": True if child.get("toggle") == "yes" else False, + "default": True if child.get("default") == "no" else True, } for child in features_nodes } + return result @@ -5623,7 +5125,7 @@ def domain_capabilities(emulator=None, arch=None, machine=None, domain=None, **k """ Return the domain capabilities given an emulator, architecture, machine or virtualization type. - .. versionadded:: Fluorine + .. versionadded:: 2019.2.0 :param emulator: return the capabilities for the given emulator binary :param arch: return the capabilities for the given CPU architecture @@ -5662,7 +5164,7 @@ def all_capabilities(**kwargs): """ Return the host and domain capabilities in a single call. - .. versionadded:: Neon + .. versionadded:: Sodium :param connection: libvirt connection URI, overriding defaults :param username: username to connect with, overriding defaults @@ -5929,7 +5431,7 @@ def list_networks(**kwargs): def network_info(name=None, **kwargs): """ - Return information on a virtual network provided its name. + Return informations on a virtual network provided its name. :param name: virtual network name :param connection: libvirt connection URI, overriding defaults @@ -6526,19 +6028,15 @@ def _pool_set_secret( if secret_type: # Get the previously defined secret if any secret = None - try: - if usage: - usage_type = ( - libvirt.VIR_SECRET_USAGE_TYPE_CEPH - if secret_type == "ceph" - else libvirt.VIR_SECRET_USAGE_TYPE_ISCSI - ) - secret = conn.secretLookupByUsage(usage_type, usage) - elif uuid: - secret = conn.secretLookupByUUIDString(uuid) - except libvirt.libvirtError as err: - # For some reason the secret has been removed. Don't fail since we'll recreate it - log.info("Secret not found: %s", err.get_error_message()) + if usage: + usage_type = ( + libvirt.VIR_SECRET_USAGE_TYPE_CEPH + if secret_type == "ceph" + else libvirt.VIR_SECRET_USAGE_TYPE_ISCSI + ) + secret = conn.secretLookupByUsage(usage_type, usage) + elif uuid: + secret = conn.secretLookupByUUIDString(uuid) # Create secret if needed if not secret: @@ -6790,7 +6288,7 @@ def list_pools(**kwargs): def pool_info(name=None, **kwargs): """ - Return information on a storage pool provided its name. + Return informations on a storage pool provided its name. :param name: libvirt storage pool name :param connection: libvirt connection URI, overriding defaults @@ -7007,6 +6505,22 @@ def pool_delete(name, **kwargs): conn = __get_conn(**kwargs) try: pool = conn.storagePoolLookupByName(name) + desc = ElementTree.fromstring(pool.XMLDesc()) + + # Is there a secret that we generated and would need to be removed? + # Don't remove the other secrets + auth_node = desc.find("source/auth") + if auth_node is not None: + auth_types = { + "ceph": libvirt.VIR_SECRET_USAGE_TYPE_CEPH, + "iscsi": libvirt.VIR_SECRET_USAGE_TYPE_ISCSI, + } + secret_type = auth_types[auth_node.get("type")] + secret_usage = auth_node.find("secret").get("usage") + if secret_type and "pool_{}".format(name) == secret_usage: + secret = conn.secretLookupByUsage(secret_type, secret_usage) + secret.undefine() + return not bool(pool.delete(libvirt.VIR_STORAGE_POOL_DELETE_NORMAL)) finally: conn.close() @@ -7125,33 +6639,29 @@ def _is_valid_volume(vol): def _get_all_volumes_paths(conn): """ - Extract the path, name, pool name and backing stores path of all volumes. + Extract the path and backing stores path of all volumes. :param conn: libvirt connection to use """ - pools = [ - pool - for pool in conn.listAllStoragePools() - if pool.info()[0] == libvirt.VIR_STORAGE_POOL_RUNNING + volumes = [ + vol + for l in [ + obj.listAllVolumes() + for obj in conn.listAllStoragePools() + if obj.info()[0] == libvirt.VIR_STORAGE_POOL_RUNNING + ] + for vol in l ] - volumes = {} - for pool in pools: - pool_volumes = { - volume.path(): { - "pool": pool.name(), - "name": volume.name(), - "backing_stores": [ - path.text - for path in ElementTree.fromstring(volume.XMLDesc()).findall( - ".//backingStore/path" - ) - ], - } - for volume in pool.listAllVolumes() - if _is_valid_volume(volume) - } - volumes.update(pool_volumes) - return volumes + return { + vol.path(): [ + path.text + for path in ElementTree.fromstring(vol.XMLDesc()).findall( + ".//backingStore/path" + ) + ] + for vol in volumes + if _is_valid_volume(vol) + } def volume_infos(pool=None, volume=None, **kwargs): @@ -7222,8 +6732,8 @@ def volume_infos(pool=None, volume=None, **kwargs): if vol.path(): as_backing_store = { path - for (path, volume) in backing_stores.items() - if vol.path() in volume.get("backing_stores") + for (path, all_paths) in backing_stores.items() + if vol.path() in all_paths } used_by = [ vm_name @@ -7346,7 +6856,7 @@ def volume_define( backing_store="{'path': '/path/to/base.img', 'format': 'raw'}" \ nocow=True - .. versionadded:: 3001 + .. versionadded:: Sodium """ ret = False try: @@ -7488,7 +6998,7 @@ def volume_upload(pool, volume, file, offset=0, length=0, sparse=False, **kwargs salt '*' virt.volume_upload default myvm.qcow2 /path/to/disk.qcow2 - .. versionadded:: 3001 + .. versionadded:: Sodium """ conn = __get_conn(**kwargs) diff --git a/salt/states/virt.py b/salt/states/virt.py index 2394d0745e..cb15d57d8f 100644 --- a/salt/states/virt.py +++ b/salt/states/virt.py @@ -33,8 +33,6 @@ except ImportError: __virtualname__ = "virt" -log = logging.getLogger(__name__) - def __virtual__(): """ @@ -42,6 +40,7 @@ def __virtual__(): :return: """ + if "virt.node_info" in __salt__: return __virtualname__ return (False, "virt module could not be loaded") @@ -332,19 +331,15 @@ def defined( but ``x86_64`` is prefed over ``i686``. Only used when creating a new virtual machine. :param boot: - Specifies kernel for the virtual machine, as well as boot parameters - for the virtual machine. This is an optionl parameter, and all of the - keys are optional within the dictionary. If a remote path is provided - to kernel or initrd, salt will handle the downloading of the specified - remote fild, and will modify the XML accordingly. + Specifies kernel, initial ramdisk and kernel command line parameters for the virtual machine. + This is an optional parameter, all of the keys are optional within the dictionary. - .. code-block:: python + Refer to :ref:`init-boot-def` for the complete boot parameters description. - { - 'kernel': '/root/f8-i386-vmlinuz', - 'initrd': '/root/f8-i386-initrd', - 'cmdline': 'console=ttyS0 ks=http://example.com/f8-i386/os/' - } + To update any boot parameters, specify the new path for each. To remove any boot parameters, + pass a None object, for instance: 'kernel': ``None``. + + .. versionadded:: 3000 :param update: set to ``False`` to prevent updating a defined domain. (Default: ``True``) @@ -461,7 +456,6 @@ def running( name, cpu=None, mem=None, - image=None, vm_type=None, disk_profile=None, disks=None, @@ -487,23 +481,7 @@ def running( :param name: name of the virtual machine to run :param cpu: number of CPUs for the virtual machine to create - :param mem: Amount of memory to allocate to the virtual machine in MiB. Since 3002, a dictionary can be used to - contain detailed configuration which support memory allocation or tuning. Supported parameters are ``boot``, - ``current``, ``max``, ``slots``, ``hard_limit``, ``soft_limit``, ``swap_hard_limit`` and ``min_guarantee``. The - structure of the dictionary is documented in :ref:`init-mem-def`. Both decimal and binary base are supported. - Detail unit specification is documented in :ref:`virt-units`. Please note that the value for ``slots`` must be - an integer. - - To remove any parameters, pass a None object, for instance: 'soft_limit': ``None``. Please note that ``None`` - is mapped to ``null`` in sls file, pass ``null`` in sls file instead. - - .. code-block:: yaml - - - mem: - hard_limit: null - soft_limit: null - - .. versionchanged:: 3002 + :param mem: amount of memory in MiB for the new virtual machine :param vm_type: force virtual machine type for the new VM. The default value is taken from the host capabilities. This could be useful for example to use ``'qemu'`` type instead of the ``'kvm'`` one. @@ -648,32 +626,10 @@ def running( """ merged_disks = disks - if image: - default_disks = [{"system": {}}] - disknames = ["system"] - if disk_profile: - disklist = copy.deepcopy( - __salt__["config.get"]("virt:disk", {}).get(disk_profile, default_disks) - ) - disknames = disklist.keys() - disk = {"name": disknames[0], "image": image} - if merged_disks: - first_disk = [d for d in merged_disks if d.get("name") == disknames[0]] - if first_disk and "image" not in first_disk[0]: - first_disk[0]["image"] = image - else: - merged_disks.append(disk) - else: - merged_disks = [disk] - salt.utils.versions.warn_until( - "Sodium", - "'image' parameter has been deprecated. Rather use the 'disks' parameter " - "to override or define the image. 'image' will be removed in {version}.", - ) if not update: salt.utils.versions.warn_until( - "Magnesium", + "Aluminium", "'update' parameter has been deprecated. Future behavior will be the one of update=True" "It will be removed in {version}.", ) @@ -1412,141 +1368,6 @@ def pool_running( # In the corner case where test=True and the pool wasn"t defined # we may get not get our pool in the info dict and that is normal. is_running = info.get(name, {}).get("state", "stopped") == "running" - if is_running: - if updated: - action = "restarted" - if not __opts__["test"]: - __salt__["virt.pool_stop"]( - name, - connection=connection, - username=username, - password=password, - ) - # if the disk or LV group is already existing build will fail (issue #56454) - if ptype in BUILDABLE_POOL_TYPES - {"disk", "logical"}: - if not __opts__["test"]: - __salt__["virt.pool_build"]( - name, - connection=connection, - username=username, - password=password, - ) - action = "built, {}".format(action) - else: - action = "already running" - result = True - - if not is_running or updated or defined: - if not __opts__["test"]: - __salt__["virt.pool_start"]( - name, - connection=connection, - username=username, - password=password, - ) - - comment = "Pool {}".format(name) - change = "Pool" - if name in ret["changes"]: - comment = "{},".format(ret["comment"]) - change = "{},".format(ret["changes"][name]) - - if action != "already running": - ret["changes"][name] = "{} {}".format(change, action) - - ret["comment"] = "{} {}".format(comment, action) - ret["result"] = result - - except libvirt.libvirtError as err: - ret["comment"] = err.get_error_message() - ret["result"] = False - - return ret - - -def pool_running( - name, - ptype=None, - target=None, - permissions=None, - source=None, - transient=False, - autostart=True, - connection=None, - username=None, - password=None, -): - """ - Defines and starts a new pool with specified arguments. - - .. versionadded:: 2019.2.0 - - :param ptype: libvirt pool type - :param target: full path to the target device or folder. (Default: ``None``) - :param permissions: - target permissions. See :ref:`pool-define-permissions` for more details on this structure. - :param source: - dictionary containing keys matching the ``source_*`` parameters in function - :func:`salt.modules.virt.pool_define`. - :param transient: - when set to ``True``, the pool will be automatically undefined after being stopped. (Default: ``False``) - :param autostart: - Whether to start the pool when booting the host. (Default: ``True``) - :param start: - When ``True``, define and start the pool, otherwise the pool will be left stopped. - :param connection: libvirt connection URI, overriding defaults - :param username: username to connect with, overriding defaults - :param password: password to connect with, overriding defaults - - .. code-block:: yaml - - pool_name: - virt.pool_running - - .. code-block:: yaml - - pool_name: - virt.pool_running: - - ptype: netfs - - target: /mnt/cifs - - permissions: - - mode: 0770 - - owner: 1000 - - group: 100 - - source: - dir: samba_share - hosts: - - one.example.com - - two.example.com - format: cifs - - autostart: True - - """ - ret = pool_defined( - name, - ptype=ptype, - target=target, - permissions=permissions, - source=source, - transient=transient, - autostart=autostart, - connection=connection, - username=username, - password=password, - ) - defined = name in ret["changes"] and ret["changes"][name].startswith("Pool defined") - updated = name in ret["changes"] and ret["changes"][name].startswith("Pool updated") - - result = True if not __opts__["test"] else None - if ret["result"] is None or ret["result"]: - try: - info = __salt__["virt.pool_info"]( - name, connection=connection, username=username, password=password - ) - action = "started" - # In the corner case where test=True and the pool wasn't defined - # we may get not get our pool in the info dict and that is normal. - is_running = info.get(name, {}).get("state", "stopped") == "running" if is_running: if updated: action = "built, restarted" @@ -1798,7 +1619,7 @@ def volume_defined( format: raw - nocow: True - .. versionadded:: 3001 + .. versionadded:: Sodium """ ret = {"name": name, "changes": {}, "result": True, "comment": ""} diff --git a/salt/templates/virt/libvirt_domain.jinja b/salt/templates/virt/libvirt_domain.jinja index da22b41440..439ed83f7f 100644 --- a/salt/templates/virt/libvirt_domain.jinja +++ b/salt/templates/virt/libvirt_domain.jinja @@ -56,13 +56,21 @@ {% if disk.type == 'file' and 'source_file' in disk -%} <source file='{{ disk.source_file }}' /> {% endif %} - {% if disk.type == 'block' -%} - <source dev='{{ disk.source_file }}' /> - {% endif %} {% if disk.type == 'volume' and 'pool' in disk -%} <source pool='{{ disk.pool }}' volume='{{ disk.volume }}' /> {% endif %} - {%- if disk.type == 'network' %}{{ libvirt_disks.network_source(disk) }}{%- endif %} + {%- if disk.type == 'network' %} + <source protocol='{{ disk.protocol }}' name='{{ disk.volume }}'{% if disk.get('query') %} query='{{ disk.query }}'{% endif %}> + {%- for host in disk.get('hosts') %} + <host name='{{ host.name }}'{% if host.get("port") %} port='{{ host.port }}'{% endif %}/> + {%- endfor %} + {%- if disk.get("auth") %} + <auth username='{{ disk.auth.username }}'> + <secret type='{{ disk.auth.type }}' usage='{{ disk.auth.usage}}'/> + </auth> + {%- endif %} + </source> + {%- endif %} <target dev='{{ disk.target_dev }}' bus='{{ disk.disk_bus }}' /> {% if disk.address -%} <address type='drive' controller='0' bus='0' target='0' unit='{{ disk.index }}' /> diff --git a/tests/unit/modules/test_virt.py b/tests/unit/modules/test_virt.py index a5c876e27d..f53b4a85c1 100644 --- a/tests/unit/modules/test_virt.py +++ b/tests/unit/modules/test_virt.py @@ -17,6 +17,7 @@ import salt.syspaths import salt.utils.yaml from salt._compat import ElementTree as ET from salt.exceptions import CommandExecutionError, SaltInvocationError +from salt.ext import six # pylint: disable=import-error from salt.ext.six.moves import range # pylint: disable=redefined-builtin @@ -1137,65 +1138,6 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): self.assertEqual("vdb2", source.attrib["volume"]) self.assertEqual("raw", disk.find("driver").get("type")) - def test_get_xml_volume_xen_dir(self): - """ - Test virt._gen_xml generating disks for a Xen hypervisor - """ - self.mock_conn.listStoragePools.return_value = ["default"] - pool_mock = MagicMock() - pool_mock.XMLDesc.return_value = ( - "<pool type='dir'><target><path>/path/to/images</path></target></pool>" - ) - volume_xml = "<volume><target><path>/path/to/images/hello_system</path></target></volume>" - pool_mock.storageVolLookupByName.return_value.XMLDesc.return_value = volume_xml - self.mock_conn.storagePoolLookupByName.return_value = pool_mock - diskp = virt._disk_profile( - self.mock_conn, - None, - "xen", - [{"name": "system", "pool": "default"}], - "hello", - ) - xml_data = virt._gen_xml( - self.mock_conn, "hello", 1, 512, diskp, [], "xen", "hvm", "x86_64", - ) - root = ET.fromstring(xml_data) - disk = root.findall(".//disk")[0] - self.assertEqual(disk.attrib["type"], "file") - self.assertEqual( - "/path/to/images/hello_system", disk.find("source").attrib["file"] - ) - - def test_get_xml_volume_xen_block(self): - """ - Test virt._gen_xml generating disks for a Xen hypervisor - """ - self.mock_conn.listStoragePools.return_value = ["default"] - pool_mock = MagicMock() - pool_mock.listVolumes.return_value = ["vol01"] - volume_xml = "<volume><target><path>/dev/to/vol01</path></target></volume>" - pool_mock.storageVolLookupByName.return_value.XMLDesc.return_value = volume_xml - self.mock_conn.storagePoolLookupByName.return_value = pool_mock - - for pool_type in ["logical", "disk", "iscsi", "scsi"]: - pool_mock.XMLDesc.return_value = "<pool type='{}'><source><device path='/dev/sda'/></source></pool>".format( - pool_type - ) - diskp = virt._disk_profile( - self.mock_conn, - None, - "xen", - [{"name": "system", "pool": "default", "source_file": "vol01"}], - "hello", - ) - xml_data = virt._gen_xml( - self.mock_conn, "hello", 1, 512, diskp, [], "xen", "hvm", "x86_64", - ) - root = ET.fromstring(xml_data) - disk = root.findall(".//disk")[0] - self.assertEqual(disk.attrib["type"], "block") - self.assertEqual("/dev/to/vol01", disk.find("source").attrib["dev"]) - def test_gen_xml_cdrom(self): """ Test virt._gen_xml(), generating a cdrom device (different disk type, no source) @@ -1849,21 +1791,21 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): self.assertEqual( { "definition": False, - "disk": {"attached": [], "detached": []}, + "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("my vm"), + virt.update("my_vm"), ) # Same parameters passed than in default virt.defined state case self.assertEqual( { "definition": False, - "disk": {"attached": [], "detached": []}, + "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, virt.update( - "my vm", + "my_vm", cpu=None, mem=None, disk_profile=None, @@ -1886,235 +1828,707 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): { "definition": True, "cpu": True, - "disk": {"attached": [], "detached": []}, + "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("my vm", cpu=2), + virt.update("my_vm", cpu=2), ) setxml = ET.fromstring(define_mock.call_args[0][0]) self.assertEqual(setxml.find("vcpu").text, "2") self.assertEqual(setvcpus_mock.call_args[0][0], 2) - setxml = ET.fromstring(define_mock_boot.call_args[0][0]) + boot = { + "kernel": "/root/f8-i386-vmlinuz", + "initrd": "/root/f8-i386-initrd", + "cmdline": "console=ttyS0 ks=http://example.com/f8-i386/os/", + } + + boot_uefi = { + "loader": "/usr/share/OVMF/OVMF_CODE.fd", + "nvram": "/usr/share/OVMF/OVMF_VARS.ms.fd", + } + + invalid_boot = { + "loader": "/usr/share/OVMF/OVMF_CODE.fd", + "initrd": "/root/f8-i386-initrd", + } + + # Update with boot parameter case self.assertEqual( - setxml.find("os").find("loader").text, "/usr/share/new/OVMF_CODE.fd" + { + "definition": True, + "disk": {"attached": [], "detached": [], "updated": []}, + "interface": {"attached": [], "detached": []}, + }, + virt.update("my_vm", boot=boot), + ) + setxml = ET.fromstring(define_mock.call_args[0][0]) + self.assertEqual(setxml.find("os").find("kernel").text, "/root/f8-i386-vmlinuz") + self.assertEqual(setxml.find("os").find("initrd").text, "/root/f8-i386-initrd") + self.assertEqual( + setxml.find("os").find("cmdline").text, + "console=ttyS0 ks=http://example.com/f8-i386/os/", + ) + setxml = ET.fromstring(define_mock.call_args[0][0]) + self.assertEqual(setxml.find("os").find("kernel").text, "/root/f8-i386-vmlinuz") + self.assertEqual(setxml.find("os").find("initrd").text, "/root/f8-i386-initrd") + self.assertEqual( + setxml.find("os").find("cmdline").text, + "console=ttyS0 ks=http://example.com/f8-i386/os/", + ) + + self.assertEqual( + { + "definition": True, + "disk": {"attached": [], "detached": [], "updated": []}, + "interface": {"attached": [], "detached": []}, + }, + virt.update("my_vm", boot=boot_uefi), + ) + setxml = ET.fromstring(define_mock.call_args[0][0]) + self.assertEqual( + setxml.find("os").find("loader").text, "/usr/share/OVMF/OVMF_CODE.fd" ) self.assertEqual(setxml.find("os").find("loader").attrib.get("readonly"), "yes") self.assertEqual(setxml.find("os").find("loader").attrib["type"], "pflash") self.assertEqual( setxml.find("os").find("nvram").attrib["template"], - "/usr/share/new/OVMF_VARS.ms.fd", + "/usr/share/OVMF/OVMF_VARS.ms.fd", ) - kernel_none = { - "kernel": None, - "initrd": None, - "cmdline": None, - } + with self.assertRaises(SaltInvocationError): + virt.update("my_vm", boot=invalid_boot) - uefi_none = {"loader": None, "nvram": None} + # Update memory case + setmem_mock = MagicMock(return_value=0) + domain_mock.setMemoryFlags = setmem_mock self.assertEqual( { "definition": True, + "mem": True, "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("vm_with_boot_param", boot=kernel_none), + virt.update("my_vm", mem=2048), ) + setxml = ET.fromstring(define_mock.call_args[0][0]) + self.assertEqual(setxml.find("memory").text, "2048") + self.assertEqual(setxml.find("memory").get("unit"), "MiB") + self.assertEqual(setmem_mock.call_args[0][0], 2048 * 1024) + + # Update disks case + devattach_mock = MagicMock(return_value=0) + devdetach_mock = MagicMock(return_value=0) + domain_mock.attachDevice = devattach_mock + domain_mock.detachDevice = devdetach_mock + mock_chmod = MagicMock() + mock_run = MagicMock() + with patch.dict( + os.__dict__, {"chmod": mock_chmod, "makedirs": MagicMock()} + ): # pylint: disable=no-member + with patch.dict( + virt.__salt__, {"cmd.run": mock_run} + ): # pylint: disable=no-member + ret = virt.update( + "my_vm", + disk_profile="default", + disks=[ + { + "name": "cddrive", + "device": "cdrom", + "source_file": None, + "model": "ide", + }, + {"name": "added", "size": 2048}, + ], + ) + added_disk_path = os.path.join( + virt.__salt__["config.get"]("virt:images"), "my_vm_added.qcow2" + ) # pylint: disable=no-member + self.assertEqual( + mock_run.call_args[0][0], + 'qemu-img create -f qcow2 "{}" 2048M'.format(added_disk_path), + ) + self.assertEqual(mock_chmod.call_args[0][0], added_disk_path) + self.assertListEqual( + [None, os.path.join(root_dir, "my_vm_added.qcow2")], + [ + ET.fromstring(disk).find("source").get("file") + if str(disk).find("<source") > -1 + else None + for disk in ret["disk"]["attached"] + ], + ) - setxml = ET.fromstring(define_mock_boot.call_args[0][0]) - self.assertEqual(setxml.find("os").find("kernel"), None) - self.assertEqual(setxml.find("os").find("initrd"), None) - self.assertEqual(setxml.find("os").find("cmdline"), None) + self.assertListEqual( + ["my_vm_data", "libvirt-pool/my_vm_data2"], + [ + ET.fromstring(disk).find("source").get("volume") + or ET.fromstring(disk).find("source").get("name") + for disk in ret["disk"]["detached"] + ], + ) + self.assertEqual(devattach_mock.call_count, 2) + self.assertEqual(devdetach_mock.call_count, 2) + # Update nics case + yaml_config = """ + virt: + nic: + myprofile: + - network: default + name: eth0 + """ + mock_config = salt.utils.yaml.safe_load(yaml_config) + devattach_mock.reset_mock() + devdetach_mock.reset_mock() + with patch.dict( + salt.modules.config.__opts__, mock_config # pylint: disable=no-member + ): + ret = virt.update( + "my_vm", + nic_profile="myprofile", + interfaces=[ + { + "name": "eth0", + "type": "network", + "source": "default", + "mac": "52:54:00:39:02:b1", + }, + {"name": "eth1", "type": "network", "source": "newnet"}, + ], + ) + self.assertEqual( + ["newnet"], + [ + ET.fromstring(nic).find("source").get("network") + for nic in ret["interface"]["attached"] + ], + ) + self.assertEqual( + ["oldnet"], + [ + ET.fromstring(nic).find("source").get("network") + for nic in ret["interface"]["detached"] + ], + ) + devattach_mock.assert_called_once() + devdetach_mock.assert_called_once() + + # Remove nics case + devattach_mock.reset_mock() + devdetach_mock.reset_mock() + ret = virt.update("my_vm", nic_profile=None, interfaces=[]) + self.assertEqual([], ret["interface"]["attached"]) + self.assertEqual(2, len(ret["interface"]["detached"])) + devattach_mock.assert_not_called() + devdetach_mock.assert_called() + + # Remove disks case (yeah, it surely is silly) + devattach_mock.reset_mock() + devdetach_mock.reset_mock() + ret = virt.update("my_vm", disk_profile=None, disks=[]) + self.assertEqual([], ret["disk"]["attached"]) + self.assertEqual(3, len(ret["disk"]["detached"])) + devattach_mock.assert_not_called() + devdetach_mock.assert_called() + + # Graphics change test case self.assertEqual( { "definition": True, "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("vm_with_boot_param", boot={"efi": False}), + virt.update("my_vm", graphics={"type": "vnc"}), ) - setxml = ET.fromstring(define_mock_boot.call_args[0][0]) - self.assertEqual(setxml.find("os").find("nvram"), None) - self.assertEqual(setxml.find("os").find("loader"), None) + setxml = ET.fromstring(define_mock.call_args[0][0]) + self.assertEqual("vnc", setxml.find("devices/graphics").get("type")) + # Update with no diff case + pool_mock = MagicMock() + default_pool_desc = "<pool type='dir'></pool>" + rbd_pool_desc = """ + <pool type='rbd'> + <name>test-rbd</name> + <source> + <host name='ses2.tf.local'/> + <host name='ses3.tf.local' port='1234'/> + <name>libvirt-pool</name> + <auth type='ceph' username='libvirt'> + <secret usage='pool_test-rbd'/> + </auth> + </source> + </pool> + """ + pool_mock.XMLDesc.side_effect = [ + default_pool_desc, + rbd_pool_desc, + default_pool_desc, + rbd_pool_desc, + ] + self.mock_conn.storagePoolLookupByName.return_value = pool_mock + self.mock_conn.listStoragePools.return_value = ["test-rbd", "default"] + self.assertEqual( + { + "definition": False, + "disk": {"attached": [], "detached": [], "updated": []}, + "interface": {"attached": [], "detached": []}, + }, + virt.update( + "my_vm", + cpu=1, + mem=1024, + disk_profile="default", + disks=[ + {"name": "data", "size": 2048, "pool": "default"}, + { + "name": "data2", + "size": 4096, + "pool": "test-rbd", + "format": "raw", + }, + ], + nic_profile="myprofile", + interfaces=[ + { + "name": "eth0", + "type": "network", + "source": "default", + "mac": "52:54:00:39:02:b1", + }, + {"name": "eth1", "type": "network", "source": "oldnet"}, + ], + graphics={ + "type": "spice", + "listen": {"type": "address", "address": "127.0.0.1"}, + }, + ), + ) + + # Failed XML description update case + self.mock_conn.defineXML.side_effect = self.mock_libvirt.libvirtError( + "Test error" + ) + setmem_mock.reset_mock() + with self.assertRaises(self.mock_libvirt.libvirtError): + virt.update("my_vm", mem=2048) + + # Failed single update failure case + self.mock_conn.defineXML = MagicMock(return_value=True) + setmem_mock.side_effect = self.mock_libvirt.libvirtError( + "Failed to live change memory" + ) self.assertEqual( { "definition": True, + "errors": ["Failed to live change memory"], "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("vm_with_boot_param", boot=uefi_none), + virt.update("my_vm", mem=2048), + ) + + # Failed multiple updates failure case + self.assertEqual( + { + "definition": True, + "errors": ["Failed to live change memory"], + "cpu": True, + "disk": {"attached": [], "detached": [], "updated": []}, + "interface": {"attached": [], "detached": []}, + }, + virt.update("my_vm", cpu=4, mem=2048), + ) + + def test_update_backing_store(self): + """ + Test updating a disk with a backing store + """ + xml = """ + <domain type='kvm' id='7'> + <name>my_vm</name> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='auto'>1</vcpu> + <os> + <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type> + </os> + <devices> + <disk type='volume' device='disk'> + <driver name='qemu' type='qcow2' cache='none' io='native'/> + <source pool='default' volume='my_vm_system' index='1'/> + <backingStore type='file' index='2'> + <format type='qcow2'/> + <source file='/path/to/base.qcow2'/> + <backingStore/> + </backingStore> + <target dev='vda' bus='virtio'/> + <alias name='virtio-disk0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + </devices> + </domain> + """ + domain_mock = self.set_mock_vm("my_vm", xml) + domain_mock.OSType.return_value = "hvm" + self.mock_conn.defineXML.return_value = True + updatedev_mock = MagicMock(return_value=0) + domain_mock.updateDeviceFlags = updatedev_mock + self.mock_conn.listStoragePools.return_value = ["default"] + self.mock_conn.storagePoolLookupByName.return_value.XMLDesc.return_value = ( + "<pool type='dir'/>" + ) + + ret = virt.update( + "my_vm", + disks=[ + { + "name": "system", + "pool": "default", + "backing_store_path": "/path/to/base.qcow2", + "backing_store_format": "qcow2", + }, + ], + ) + self.assertFalse(ret["definition"]) + self.assertFalse(ret["disk"]["attached"]) + self.assertFalse(ret["disk"]["detached"]) + + def test_update_removables(self): + """ + Test attaching, detaching, changing removable devices + """ + xml = """ + <domain type='kvm' id='7'> + <name>my_vm</name> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='auto'>1</vcpu> + <os> + <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type> + </os> + <devices> + <disk type='network' device='cdrom'> + <driver name='qemu' type='raw' cache='none' io='native'/> + <source protocol='https' name='/dvd-image-1.iso'> + <host name='test-srv.local' port='80'/> + </source> + <backingStore/> + <target dev='hda' bus='ide'/> + <readonly/> + <alias name='ide0-0-0'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <disk type='file' device='cdrom'> + <driver name='qemu' type='raw' cache='none' io='native'/> + <target dev='hdb' bus='ide'/> + <readonly/> + <alias name='ide0-0-1'/> + <address type='drive' controller='0' bus='0' target='0' unit='1'/> + </disk> + <disk type='file' device='cdrom'> + <driver name='qemu' type='raw' cache='none' io='native'/> + <source file='/srv/dvd-image-2.iso'/> + <backingStore/> + <target dev='hdc' bus='ide'/> + <readonly/> + <alias name='ide0-0-2'/> + <address type='drive' controller='0' bus='0' target='0' unit='2'/> + </disk> + <disk type='file' device='cdrom'> + <driver name='qemu' type='raw' cache='none' io='native'/> + <source file='/srv/dvd-image-3.iso'/> + <backingStore/> + <target dev='hdd' bus='ide'/> + <readonly/> + <alias name='ide0-0-3'/> + <address type='drive' controller='0' bus='0' target='0' unit='3'/> + </disk> + <disk type='network' device='cdrom'> + <driver name='qemu' type='raw' cache='none' io='native'/> + <source protocol='https' name='/dvd-image-6.iso'> + <host name='test-srv.local' port='80'/> + </source> + <backingStore/> + <target dev='hde' bus='ide'/> + <readonly/> + </disk> + </devices> + </domain> + """ + domain_mock = self.set_mock_vm("my_vm", xml) + domain_mock.OSType.return_value = "hvm" + self.mock_conn.defineXML.return_value = True + updatedev_mock = MagicMock(return_value=0) + domain_mock.updateDeviceFlags = updatedev_mock + + ret = virt.update( + "my_vm", + disks=[ + { + "name": "dvd1", + "device": "cdrom", + "source_file": None, + "model": "ide", + }, + { + "name": "dvd2", + "device": "cdrom", + "source_file": "/srv/dvd-image-4.iso", + "model": "ide", + }, + { + "name": "dvd3", + "device": "cdrom", + "source_file": "/srv/dvd-image-2.iso", + "model": "ide", + }, + { + "name": "dvd4", + "device": "cdrom", + "source_file": "/srv/dvd-image-5.iso", + "model": "ide", + }, + { + "name": "dvd5", + "device": "cdrom", + "source_file": "/srv/dvd-image-6.iso", + "model": "ide", + }, + ], + ) + + self.assertTrue(ret["definition"]) + self.assertFalse(ret["disk"]["attached"]) + self.assertFalse(ret["disk"]["detached"]) + self.assertEqual( + [ + { + "type": "file", + "device": "cdrom", + "driver": { + "name": "qemu", + "type": "raw", + "cache": "none", + "io": "native", + }, + "backingStore": None, + "target": {"dev": "hda", "bus": "ide"}, + "readonly": None, + "alias": {"name": "ide0-0-0"}, + "address": { + "type": "drive", + "controller": "0", + "bus": "0", + "target": "0", + "unit": "0", + }, + }, + { + "type": "file", + "device": "cdrom", + "driver": { + "name": "qemu", + "type": "raw", + "cache": "none", + "io": "native", + }, + "target": {"dev": "hdb", "bus": "ide"}, + "readonly": None, + "alias": {"name": "ide0-0-1"}, + "address": { + "type": "drive", + "controller": "0", + "bus": "0", + "target": "0", + "unit": "1", + }, + "source": {"file": "/srv/dvd-image-4.iso"}, + }, + { + "type": "file", + "device": "cdrom", + "driver": { + "name": "qemu", + "type": "raw", + "cache": "none", + "io": "native", + }, + "backingStore": None, + "target": {"dev": "hdd", "bus": "ide"}, + "readonly": None, + "alias": {"name": "ide0-0-3"}, + "address": { + "type": "drive", + "controller": "0", + "bus": "0", + "target": "0", + "unit": "3", + }, + "source": {"file": "/srv/dvd-image-5.iso"}, + }, + { + "type": "file", + "device": "cdrom", + "driver": { + "name": "qemu", + "type": "raw", + "cache": "none", + "io": "native", + }, + "backingStore": None, + "target": {"dev": "hde", "bus": "ide"}, + "readonly": None, + "source": {"file": "/srv/dvd-image-6.iso"}, + }, + ], + [ + salt.utils.xmlutil.to_dict(ET.fromstring(disk), True) + for disk in ret["disk"]["updated"] + ], ) - setxml = ET.fromstring(define_mock_boot.call_args[0][0]) - self.assertEqual(setxml.find("os").find("loader"), None) - self.assertEqual(setxml.find("os").find("nvram"), None) - - def test_update_memtune_params(self): + def test_update_existing_boot_params(self): """ - Test virt.update() with memory tuning parameters. + Test virt.update() with existing boot parameters. """ - xml_with_memtune_params = """ + root_dir = os.path.join(salt.syspaths.ROOT_DIR, "srv", "salt-images") + xml_boot = """ <domain type='kvm' id='8'> <name>vm_with_boot_param</name> <memory unit='KiB'>1048576</memory> <currentMemory unit='KiB'>1048576</currentMemory> - <maxMemory slots="12" unit="bytes">1048576</maxMemory> <vcpu placement='auto'>1</vcpu> - <memtune> - <hard_limit unit="KiB">1048576</hard_limit> - <soft_limit unit="KiB">2097152</soft_limit> - <swap_hard_limit unit="KiB">2621440</swap_hard_limit> - <min_guarantee unit='KiB'>671088</min_guarantee> - </memtune> <os> <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type> + <kernel>/boot/oldkernel</kernel> + <initrd>/boot/initrdold.img</initrd> + <cmdline>console=ttyS0 ks=http://example.com/old/os/</cmdline> + <loader>/usr/share/old/OVMF_CODE.fd</loader> + <nvram>/usr/share/old/OVMF_VARS.ms.fd</nvram> </os> + <devices> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='{0}{1}vm_with_boot_param_system.qcow2'/> + <backingStore/> + <target dev='vda' bus='virtio'/> + <alias name='virtio-disk0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> + </disk> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='{0}{1}vm_with_boot_param_data.qcow2'/> + <backingStore/> + <target dev='vdb' bus='virtio'/> + <alias name='virtio-disk1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> + </disk> + <interface type='network'> + <mac address='52:54:00:39:02:b1'/> + <source network='default' bridge='virbr0'/> + <target dev='vnet0'/> + <model type='virtio'/> + <alias name='net0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + <interface type='network'> + <mac address='52:54:00:39:02:b2'/> + <source network='oldnet' bridge='virbr1'/> + <target dev='vnet1'/> + <model type='virtio'/> + <alias name='net1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/> + </interface> + <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'> + <listen type='address' address='127.0.0.1'/> + </graphics> + <video> + <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> + <alias name='video0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </video> + </devices> </domain> - """ - domain_mock = self.set_mock_vm("vm_with_memtune_param", xml_with_memtune_params) - domain_mock.OSType = MagicMock(return_value="hvm") - define_mock = MagicMock(return_value=True) - self.mock_conn.defineXML = define_mock + """.format( + root_dir, os.sep + ) + domain_mock_boot = self.set_mock_vm("vm_with_boot_param", xml_boot) + domain_mock_boot.OSType = MagicMock(return_value="hvm") + define_mock_boot = MagicMock(return_value=True) + self.mock_conn.defineXML = define_mock_boot + boot_new = { + "kernel": "/root/new-vmlinuz", + "initrd": "/root/new-initrd", + "cmdline": "console=ttyS0 ks=http://example.com/new/os/", + } - memtune_new_val = { - "boot": "0.7g", - "current": "2.5g", - "max": "3096m", - "slots": "10", - "soft_limit": "2048m", - "hard_limit": "1024", - "swap_hard_limit": "2.5g", - "min_guarantee": "1 g", + uefi_boot_new = { + "loader": "/usr/share/new/OVMF_CODE.fd", + "nvram": "/usr/share/new/OVMF_VARS.ms.fd", } - domain_mock.setMemoryFlags.return_value = 0 self.assertEqual( { "definition": True, "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, - "mem": True, }, - virt.update("vm_with_memtune_param", mem=memtune_new_val), - ) - self.assertEqual( - domain_mock.setMemoryFlags.call_args[0][0], int(2.5 * 1024 ** 2) - ) - - setxml = ET.fromstring(define_mock.call_args[0][0]) - self.assertEqual( - setxml.find("memtune").find("soft_limit").text, str(2048 * 1024) + virt.update("vm_with_boot_param", boot=boot_new), ) + setxml_boot = ET.fromstring(define_mock_boot.call_args[0][0]) self.assertEqual( - setxml.find("memtune").find("hard_limit").text, str(1024 * 1024) + setxml_boot.find("os").find("kernel").text, "/root/new-vmlinuz" ) + self.assertEqual(setxml_boot.find("os").find("initrd").text, "/root/new-initrd") self.assertEqual( - setxml.find("memtune").find("swap_hard_limit").text, - str(int(2.5 * 1024 ** 2)), + setxml_boot.find("os").find("cmdline").text, + "console=ttyS0 ks=http://example.com/new/os/", ) - self.assertEqual( - setxml.find("memtune").find("swap_hard_limit").get("unit"), "KiB", - ) - self.assertEqual( - setxml.find("memtune").find("min_guarantee").text, str(1 * 1024 ** 3) - ) - self.assertEqual( - setxml.find("memtune").find("min_guarantee").attrib.get("unit"), "bytes" - ) - self.assertEqual(setxml.find("maxMemory").text, str(3096 * 1024 ** 2)) - self.assertEqual(setxml.find("maxMemory").attrib.get("slots"), "10") - self.assertEqual(setxml.find("currentMemory").text, str(int(2.5 * 1024 ** 3))) - self.assertEqual(setxml.find("memory").text, str(int(0.7 * 1024 ** 3))) - - max_slot_reverse = { - "slots": "10", - "max": "3096m", - } - self.assertEqual( - { - "definition": True, - "disk": {"attached": [], "detached": [], "updated": []}, - "interface": {"attached": [], "detached": []}, - }, - virt.update("vm_with_memtune_param", mem=max_slot_reverse), - ) - setxml = ET.fromstring(define_mock.call_args[0][0]) - self.assertEqual(setxml.find("maxMemory").text, str(3096 * 1024 ** 2)) - self.assertEqual(setxml.find("maxMemory").get("unit"), "bytes") - self.assertEqual(setxml.find("maxMemory").attrib.get("slots"), "10") - - max_swap_none = { - "boot": "0.7g", - "current": "2.5g", - "max": None, - "slots": "10", - "soft_limit": "2048m", - "hard_limit": "1024", - "swap_hard_limit": None, - "min_guarantee": "1 g", - } - domain_mock.setMemoryFlags.reset_mock() self.assertEqual( { "definition": True, "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, - "mem": True, }, - virt.update("vm_with_memtune_param", mem=max_swap_none), - ) - self.assertEqual( - domain_mock.setMemoryFlags.call_args[0][0], int(2.5 * 1024 ** 2) + virt.update("vm_with_boot_param", boot=uefi_boot_new), ) - setxml = ET.fromstring(define_mock.call_args[0][0]) - self.assertEqual( - setxml.find("memtune").find("soft_limit").text, str(2048 * 1024) - ) - self.assertEqual( - setxml.find("memtune").find("hard_limit").text, str(1024 * 1024) - ) - self.assertEqual(setxml.find("memtune").find("swap_hard_limit"), None) + setxml = ET.fromstring(define_mock_boot.call_args[0][0]) self.assertEqual( - setxml.find("memtune").find("min_guarantee").text, str(1 * 1024 ** 3) + setxml.find("os").find("loader").text, "/usr/share/new/OVMF_CODE.fd" ) + self.assertEqual(setxml.find("os").find("loader").attrib.get("readonly"), "yes") + self.assertEqual(setxml.find("os").find("loader").attrib["type"], "pflash") self.assertEqual( - setxml.find("memtune").find("min_guarantee").attrib.get("unit"), "bytes" + setxml.find("os").find("nvram").attrib["template"], + "/usr/share/new/OVMF_VARS.ms.fd", ) - self.assertEqual(setxml.find("maxMemory").text, None) - self.assertEqual(setxml.find("currentMemory").text, str(int(2.5 * 1024 ** 3))) - self.assertEqual(setxml.find("memory").text, str(int(0.7 * 1024 ** 3))) - memtune_none = { - "soft_limit": None, - "hard_limit": None, - "swap_hard_limit": None, - "min_guarantee": None, + kernel_none = { + "kernel": None, + "initrd": None, + "cmdline": None, } + uefi_none = {"loader": None, "nvram": None} + self.assertEqual( { "definition": True, "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("vm_with_memtune_param", mem=memtune_none), + virt.update("vm_with_boot_param", boot=kernel_none), ) - setxml = ET.fromstring(define_mock.call_args[0][0]) - self.assertEqual(setxml.find("memtune").find("soft_limit"), None) - self.assertEqual(setxml.find("memtune").find("hard_limit"), None) - self.assertEqual(setxml.find("memtune").find("swap_hard_limit"), None) - self.assertEqual(setxml.find("memtune").find("min_guarantee"), None) - - max_none = { - "max": None, - } + setxml = ET.fromstring(define_mock_boot.call_args[0][0]) + self.assertEqual(setxml.find("os").find("kernel"), None) + self.assertEqual(setxml.find("os").find("initrd"), None) + self.assertEqual(setxml.find("os").find("cmdline"), None) self.assertEqual( { @@ -2122,48 +2536,12 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): "disk": {"attached": [], "detached": [], "updated": []}, "interface": {"attached": [], "detached": []}, }, - virt.update("vm_with_memtune_param", mem=max_none), + virt.update("vm_with_boot_param", boot=uefi_none), ) - setxml = ET.fromstring(define_mock.call_args[0][0]) - self.assertEqual(setxml.find("maxMemory"), None) - self.assertEqual(setxml.find("currentMemory").text, str(int(1 * 1024 ** 2))) - self.assertEqual(setxml.find("memory").text, str(int(1 * 1024 ** 2))) - - def test_handle_unit(self): - """ - Test regex function for handling units - """ - valid_case = [ - ("2", 2097152), - ("42", 44040192), - ("5b", 5), - ("2.3Kib", 2355), - ("5.8Kb", 5800), - ("16MiB", 16777216), - ("20 GB", 20000000000), - ("16KB", 16000), - (".5k", 512), - ("2.k", 2048), - ] - - for key, val in valid_case: - self.assertEqual(virt._handle_unit(key), val) - - invalid_case = [ - ("9ib", "invalid units"), - ("8byte", "invalid units"), - ("512bytes", "invalid units"), - ("4 Kbytes", "invalid units"), - ("3.4.MB", "invalid number"), - ("", "invalid number"), - ("bytes", "invalid number"), - ("2HB", "invalid units"), - ] - - for key, val in invalid_case: - with self.assertRaises(SaltInvocationError): - virt._handle_unit(key) + setxml = ET.fromstring(define_mock_boot.call_args[0][0]) + self.assertEqual(setxml.find("os").find("loader"), None) + self.assertEqual(setxml.find("os").find("nvram"), None) def test_mixed_dict_and_list_as_profile_objects(self): """ @@ -2242,30 +2620,6 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): self.assertEqual("/foo/bar", loader["path"]) self.assertEqual("yes", loader["readonly"]) - def test_cpu_baseline(self): - """ - Test virt.cpu_baseline() - """ - capabilities_xml = dedent( - """<capabilities> - <host> - <uuid>44454c4c-3400-105a-8033-b3c04f4b344a</uuid> - <cpu> - <arch>x86_64</arch> - <vendor>Intel</vendor> - </cpu> - </host> - </capabilities>""" - ) - - baseline_cpu_xml = b"""<cpu match="exact" mode="custom"> - <vendor>Intel</vendor> - </cpu>""" - - self.mock_conn.getCapabilities.return_value = capabilities_xml - self.mock_conn.baselineCPU.return_value = baseline_cpu_xml - self.assertMultiLineEqual(str(baseline_cpu_xml), str(virt.cpu_baseline())) - def test_parse_qemu_img_info(self): """ Make sure that qemu-img info output is properly parsed @@ -2906,7 +3260,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): "cpuselection": {"default": True, "toggle": False}, "deviceboot": {"default": True, "toggle": False}, "disksnapshot": {"default": True, "toggle": False}, - "acpi": {"default": False, "toggle": True}, + "acpi": {"default": True, "toggle": True}, "apic": {"default": True, "toggle": False}, "pae": {"default": True, "toggle": False}, "nonpae": {"default": True, "toggle": False}, @@ -2944,7 +3298,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): "deviceboot": {"default": True, "toggle": False}, "disksnapshot": {"default": True, "toggle": False}, "acpi": {"default": True, "toggle": True}, - "apic": {"default": False, "toggle": False}, + "apic": {"default": True, "toggle": False}, }, }, { @@ -3230,7 +3584,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): "44454c4c-3400-105a-8033-b3c04f4b344a", caps["host"]["host"]["uuid"] ) self.assertEqual( - {"qemu", "kvm"}, {domainCaps["domain"] for domainCaps in caps["domains"]} + {"qemu", "kvm"}, {domainCaps["domain"] for domainCaps in caps["domains"]}, ) def test_network_tag(self): @@ -3894,6 +4248,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): """ mock_pool = MagicMock() mock_pool.delete = MagicMock(return_value=0) + mock_pool.XMLDesc.return_value = "<pool type='dir'/>" self.mock_conn.storagePoolLookupByName = MagicMock(return_value=mock_pool) res = virt.pool_delete("test-pool") @@ -3907,12 +4262,12 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): self.mock_libvirt.VIR_STORAGE_POOL_DELETE_NORMAL ) - def test_pool_undefine_secret(self): + def test_pool_delete_secret(self): """ - Test virt.pool_undefine function where the pool has a secret + Test virt.pool_delete function where the pool has a secret """ mock_pool = MagicMock() - mock_pool.undefine = MagicMock(return_value=0) + mock_pool.delete = MagicMock(return_value=0) mock_pool.XMLDesc.return_value = """ <pool type='rbd'> <name>test-ses</name> @@ -3929,11 +4284,16 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): mock_undefine = MagicMock(return_value=0) self.mock_conn.secretLookupByUsage.return_value.undefine = mock_undefine - res = virt.pool_undefine("test-ses") + res = virt.pool_delete("test-ses") self.assertTrue(res) self.mock_conn.storagePoolLookupByName.assert_called_once_with("test-ses") - mock_pool.undefine.assert_called_once_with() + + # Shouldn't be called with another parameter so far since those are not implemented + # and thus throwing exceptions. + mock_pool.delete.assert_called_once_with( + self.mock_libvirt.VIR_STORAGE_POOL_DELETE_NORMAL + ) self.mock_conn.secretLookupByUsage.assert_called_once_with( self.mock_libvirt.VIR_SECRET_USAGE_TYPE_CEPH, "pool_test-ses" @@ -4202,6 +4562,24 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): </source> </pool>""" + expected_xml = ( + '<pool type="rbd">' + "<name>default</name>" + "<uuid>20fbe05c-ab40-418a-9afa-136d512f0ede</uuid>" + '<capacity unit="bytes">1999421108224</capacity>' + '<allocation unit="bytes">713207042048</allocation>' + '<available unit="bytes">1286214066176</available>' + "<source>" + '<host name="ses4.tf.local" />' + '<host name="ses5.tf.local" />' + '<auth type="ceph" username="libvirt">' + '<secret uuid="14e9a0f1-8fbf-4097-b816-5b094c182212" />' + "</auth>" + "<name>iscsi-images</name>" + "</source>" + "</pool>" + ) + mock_secret = MagicMock() self.mock_conn.secretLookupByUUIDString = MagicMock(return_value=mock_secret) @@ -4222,23 +4600,6 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): self.mock_conn.storagePoolDefineXML.assert_not_called() mock_secret.setValue.assert_called_once_with(b"secret") - # Case where the secret can't be found - self.mock_conn.secretLookupByUUIDString = MagicMock( - side_effect=self.mock_libvirt.libvirtError("secret not found") - ) - self.assertFalse( - virt.pool_update( - "default", - "rbd", - source_name="iscsi-images", - source_hosts=["ses4.tf.local", "ses5.tf.local"], - source_auth={"username": "libvirt", "password": "c2VjcmV0"}, - ) - ) - self.mock_conn.storagePoolDefineXML.assert_not_called() - self.mock_conn.secretDefineXML.assert_called_once() - mock_secret.setValue.assert_called_once_with(b"secret") - def test_pool_update_password_create(self): """ Test the pool_update function, where the password only is changed @@ -4898,3 +5259,279 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin): "vm2", inactive=False, ) + + def test_volume_define(self): + """ + Test virt.volume_define function + """ + # Normal test case + pool_mock = MagicMock() + pool_mock.XMLDesc.return_value = "<pool type='dir'></pool>" + self.mock_conn.storagePoolLookupByName.return_value = pool_mock + + self.assertTrue( + virt.volume_define( + "testpool", + "myvm_system.qcow2", + 8192, + allocation=4096, + format="qcow2", + type="file", + ) + ) + + expected_xml = ( + "<volume type='file'>\n" + " <name>myvm_system.qcow2</name>\n" + " <source>\n" + " </source>\n" + " <capacity unit='KiB'>8388608</capacity>\n" + " <allocation unit='KiB'>4194304</allocation>\n" + " <target>\n" + " <format type='qcow2'/>\n" + " </target>\n" + "</volume>" + ) + + pool_mock.createXML.assert_called_once_with(expected_xml, 0) + + # backing store test case + pool_mock.reset_mock() + self.assertTrue( + virt.volume_define( + "testpool", + "myvm_system.qcow2", + 8192, + allocation=4096, + format="qcow2", + type="file", + backing_store={"path": "/path/to/base.raw", "format": "raw"}, + ) + ) + + expected_xml = ( + "<volume type='file'>\n" + " <name>myvm_system.qcow2</name>\n" + " <source>\n" + " </source>\n" + " <capacity unit='KiB'>8388608</capacity>\n" + " <allocation unit='KiB'>4194304</allocation>\n" + " <target>\n" + " <format type='qcow2'/>\n" + " </target>\n" + " <backingStore>\n" + " <path>/path/to/base.raw</path>\n" + " <format type='raw'/>\n" + " </backingStore>\n" + "</volume>" + ) + + pool_mock.createXML.assert_called_once_with(expected_xml, 0) + + # logical pool test case + pool_mock.reset_mock() + pool_mock.XMLDesc.return_value = "<pool type='logical'></pool>" + self.mock_conn.storagePoolLookupByName.return_value = pool_mock + + self.assertTrue( + virt.volume_define( + "testVG", + "myvm_system", + 8192, + backing_store={"path": "/dev/testVG/base"}, + ) + ) + + expected_xml = ( + "<volume>\n" + " <name>myvm_system</name>\n" + " <source>\n" + " </source>\n" + " <capacity unit='KiB'>8388608</capacity>\n" + " <allocation unit='KiB'>8388608</allocation>\n" + " <target>\n" + " </target>\n" + " <backingStore>\n" + " <path>/dev/testVG/base</path>\n" + " </backingStore>\n" + "</volume>" + ) + + pool_mock.createXML.assert_called_once_with(expected_xml, 0) + + def test_volume_upload(self): + """ + Test virt.volume_upload function + """ + pool_mock = MagicMock() + vol_mock = MagicMock() + pool_mock.storageVolLookupByName.return_value = vol_mock + self.mock_conn.storagePoolLookupByName.return_value = pool_mock + stream_mock = MagicMock() + self.mock_conn.newStream.return_value = stream_mock + + open_mock = MagicMock() + close_mock = MagicMock() + with patch.dict( + os.__dict__, {"open": open_mock, "close": close_mock} + ): # pylint: disable=no-member + # Normal case + self.assertTrue(virt.volume_upload("pool0", "vol1.qcow2", "/path/to/file")) + stream_mock.sendAll.assert_called_once() + stream_mock.finish.assert_called_once() + self.mock_conn.close.assert_called_once() + vol_mock.upload.assert_called_once_with(stream_mock, 0, 0, 0) + + # Sparse upload case + stream_mock.sendAll.reset_mock() + vol_mock.upload.reset_mock() + self.assertTrue( + virt.volume_upload( + "pool0", + "vol1.qcow2", + "/path/to/file", + offset=123, + length=456, + sparse=True, + ) + ) + stream_mock.sendAll.assert_not_called() + stream_mock.sparseSendAll.assert_called_once() + vol_mock.upload.assert_called_once_with( + stream_mock, + 123, + 456, + self.mock_libvirt.VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM, + ) + + # Upload unsupported case + vol_mock.upload.side_effect = self.mock_libvirt.libvirtError("Unsupported") + self.assertRaisesRegex( + CommandExecutionError, + "Unsupported", + virt.volume_upload, + "pool0", + "vol1.qcow2", + "/path/to/file", + ) + + def test_get_disks(self): + """ + Test the virt.get_disks function + """ + # test with volumes + vm_def = """<domain type='kvm' id='3'> + <name>srv01</name> + <devices> + <disk type='volume' device='disk'> + <driver name='qemu' type='qcow2' cache='none' io='native'/> + <source pool='default' volume='srv01_system'/> + <backingStore/> + <target dev='vda' bus='virtio'/> + <alias name='virtio-disk0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <disk type='volume' device='disk'> + <driver name='qemu' type='qcow2' cache='none' io='native'/> + <source pool='default' volume='srv01_data'/> + <backingStore type='file' index='1'> + <format type='qcow2'/> + <source file='/var/lib/libvirt/images/vol01'/> + <backingStore/> + </backingStore> + <target dev='vdb' bus='virtio'/> + <alias name='virtio-disk1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </disk> + <disk type='volume' device='disk'> + <driver name='qemu' type='qcow2' cache='none' io='native'/> + <source pool='default' volume='vm05_system'/> + <backingStore type='file' index='1'> + <format type='qcow2'/> + <source file='/var/lib/libvirt/images/vm04_system.qcow2'/> + <backingStore type='file' index='2'> + <format type='raw'/> + <source file='/var/testsuite-data/disk-image-template.raw'/> + <backingStore/> + </backingStore> + </backingStore> + <target dev='vdc' bus='virtio'/> + <alias name='virtio-disk0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <disk type='network' device='cdrom'> + <driver name='qemu' type='raw' cache='none' io='native'/> + <source protocol='http' name='/pub/iso/myimage.iso' query='foo=bar&baz=flurb' index='1'> + <host name='dev-srv.tf.local' port='80'/> + </source> + <target dev='hda' bus='ide'/> + <readonly/> + <alias name='ide0-0-0'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + </devices> + </domain> + """ + self.set_mock_vm("srv01", vm_def) + + pool_mock = MagicMock() + pool_mock.storageVolLookupByName.return_value.info.return_value = [ + 0, + 1234567, + 12345, + ] + pool_mock.storageVolLookupByName.return_value.XMLDesc.side_effect = [ + "<volume />", + """ + <volume> + <backingStore> + <path>/var/lib/libvirt/images/vol01</path> + <format type="qcow2"/> + </backingStore> + </volume>""", + ] + self.mock_conn.storagePoolLookupByName.return_value = pool_mock + + self.assertDictEqual( + virt.get_disks("srv01"), + { + "vda": { + "type": "disk", + "file": "default/srv01_system", + "file format": "qcow2", + "disk size": 12345, + "virtual size": 1234567, + }, + "vdb": { + "type": "disk", + "file": "default/srv01_data", + "file format": "qcow2", + "disk size": 12345, + "virtual size": 1234567, + "backing file": { + "file": "/var/lib/libvirt/images/vol01", + "file format": "qcow2", + }, + }, + "vdc": { + "type": "disk", + "file": "default/vm05_system", + "file format": "qcow2", + "disk size": 12345, + "virtual size": 1234567, + "backing file": { + "file": "/var/lib/libvirt/images/vm04_system.qcow2", + "file format": "qcow2", + "backing file": { + "file": "/var/testsuite-data/disk-image-template.raw", + "file format": "raw", + }, + }, + }, + "hda": { + "type": "cdrom", + "file format": "raw", + "file": "http://dev-srv.tf.local:80/pub/iso/myimage.iso?foo=bar&baz=flurb", + }, + }, + ) diff --git a/tests/unit/states/test_virt.py b/tests/unit/states/test_virt.py index 0a47562074..6d38829870 100644 --- a/tests/unit/states/test_virt.py +++ b/tests/unit/states/test_virt.py @@ -8,6 +8,7 @@ import tempfile import salt.states.virt as virt import salt.utils.files from salt.exceptions import CommandExecutionError, SaltInvocationError +from salt.ext import six from tests.support.mixins import LoaderModuleMockMixin from tests.support.mock import MagicMock, mock_open, patch from tests.support.runtests import RUNTIME_VARS @@ -263,375 +264,6 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin): ret, ) - def test_defined(self): - """ - defined state test cases. - """ - ret = { - "name": "myvm", - "changes": {}, - "result": True, - "comment": "myvm is running", - } - with patch.dict(virt.__opts__, {"test": False}): - # no change test - init_mock = MagicMock(return_value=True) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": MagicMock(return_value={"definition": False}), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": False}}, - "comment": "Domain myvm unchanged", - } - ) - self.assertDictEqual(virt.defined("myvm"), ret) - - # Test defining a guest with connection details - init_mock.reset_mock() - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=[]), - "virt.init": init_mock, - "virt.update": MagicMock( - side_effect=CommandExecutionError("not found") - ), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True}}, - "comment": "Domain myvm defined", - } - ) - disks = [ - { - "name": "system", - "size": 8192, - "overlay_image": True, - "pool": "default", - "image": "/path/to/image.qcow2", - }, - {"name": "data", "size": 16834}, - ] - ifaces = [ - {"name": "eth0", "mac": "01:23:45:67:89:AB"}, - {"name": "eth1", "type": "network", "source": "admin"}, - ] - graphics = { - "type": "spice", - "listen": {"type": "address", "address": "192.168.0.1"}, - } - self.assertDictEqual( - virt.defined( - "myvm", - cpu=2, - mem=2048, - boot_dev="cdrom hd", - os_type="linux", - arch="i686", - vm_type="qemu", - disk_profile="prod", - disks=disks, - nic_profile="prod", - interfaces=ifaces, - graphics=graphics, - seed=False, - install=False, - pub_key="/path/to/key.pub", - priv_key="/path/to/key", - connection="someconnection", - username="libvirtuser", - password="supersecret", - ), - ret, - ) - init_mock.assert_called_with( - "myvm", - cpu=2, - mem=2048, - boot_dev="cdrom hd", - os_type="linux", - arch="i686", - disk="prod", - disks=disks, - nic="prod", - interfaces=ifaces, - graphics=graphics, - hypervisor="qemu", - seed=False, - boot=None, - install=False, - start=False, - pub_key="/path/to/key.pub", - priv_key="/path/to/key", - connection="someconnection", - username="libvirtuser", - password="supersecret", - ) - - # Working update case when running - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": MagicMock( - return_value={"definition": True, "cpu": True} - ), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True, "cpu": True}}, - "result": True, - "comment": "Domain myvm updated", - } - ) - self.assertDictEqual(virt.defined("myvm", cpu=2), ret) - - # Working update case when running with boot params - boot = { - "kernel": "/root/f8-i386-vmlinuz", - "initrd": "/root/f8-i386-initrd", - "cmdline": "console=ttyS0 ks=http://example.com/f8-i386/os/", - } - - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": MagicMock( - return_value={"definition": True, "cpu": True} - ), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True, "cpu": True}}, - "result": True, - "comment": "Domain myvm updated", - } - ) - self.assertDictEqual(virt.defined("myvm", boot=boot), ret) - - # Working update case when stopped - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": MagicMock(return_value={"definition": True}), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True}}, - "result": True, - "comment": "Domain myvm updated", - } - ) - self.assertDictEqual(virt.defined("myvm", cpu=2), ret) - - # Failed live update case - update_mock = MagicMock( - return_value={ - "definition": True, - "cpu": False, - "errors": ["some error"], - } - ) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": update_mock, - }, - ): - ret.update( - { - "changes": { - "myvm": { - "definition": True, - "cpu": False, - "errors": ["some error"], - } - }, - "result": True, - "comment": "Domain myvm updated with live update(s) failures", - } - ) - self.assertDictEqual( - virt.defined("myvm", cpu=2, boot_dev="cdrom hd"), ret - ) - update_mock.assert_called_with( - "myvm", - cpu=2, - boot_dev="cdrom hd", - mem=None, - disk_profile=None, - disks=None, - nic_profile=None, - interfaces=None, - graphics=None, - live=True, - connection=None, - username=None, - password=None, - boot=None, - test=False, - ) - - # Failed definition update case - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": MagicMock( - side_effect=[self.mock_libvirt.libvirtError("error message")] - ), - }, - ): - ret.update({"changes": {}, "result": False, "comment": "error message"}) - self.assertDictEqual(virt.defined("myvm", cpu=2), ret) - - # Test dry-run mode - with patch.dict(virt.__opts__, {"test": True}): - # Guest defined case - init_mock = MagicMock(return_value=True) - update_mock = MagicMock(side_effect=CommandExecutionError("not found")) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=[]), - "virt.init": init_mock, - "virt.update": update_mock, - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True}}, - "result": None, - "comment": "Domain myvm defined", - } - ) - disks = [ - { - "name": "system", - "size": 8192, - "overlay_image": True, - "pool": "default", - "image": "/path/to/image.qcow2", - }, - {"name": "data", "size": 16834}, - ] - ifaces = [ - {"name": "eth0", "mac": "01:23:45:67:89:AB"}, - {"name": "eth1", "type": "network", "source": "admin"}, - ] - graphics = { - "type": "spice", - "listen": {"type": "address", "address": "192.168.0.1"}, - } - self.assertDictEqual( - virt.defined( - "myvm", - cpu=2, - mem=2048, - os_type="linux", - arch="i686", - vm_type="qemu", - disk_profile="prod", - disks=disks, - nic_profile="prod", - interfaces=ifaces, - graphics=graphics, - seed=False, - install=False, - pub_key="/path/to/key.pub", - priv_key="/path/to/key", - connection="someconnection", - username="libvirtuser", - password="supersecret", - ), - ret, - ) - init_mock.assert_not_called() - update_mock.assert_not_called() - - # Guest update case - update_mock = MagicMock(return_value={"definition": True}) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": update_mock, - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True}}, - "result": None, - "comment": "Domain myvm updated", - } - ) - self.assertDictEqual(virt.defined("myvm", cpu=2), ret) - update_mock.assert_called_with( - "myvm", - cpu=2, - mem=None, - disk_profile=None, - disks=None, - nic_profile=None, - interfaces=None, - graphics=None, - live=True, - connection=None, - username=None, - password=None, - boot=None, - test=True, - boot_dev=None, - ) - - # No changes case - update_mock = MagicMock(return_value={"definition": False}) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.list_domains": MagicMock(return_value=["myvm"]), - "virt.update": update_mock, - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": False}}, - "result": True, - "comment": "Domain myvm unchanged", - } - ) - self.assertDictEqual(virt.defined("myvm"), ret) - update_mock.assert_called_with( - "myvm", - cpu=None, - mem=None, - disk_profile=None, - disks=None, - nic_profile=None, - interfaces=None, - graphics=None, - live=True, - connection=None, - username=None, - password=None, - boot=None, - test=True, - boot_dev=None, - ) - def test_defined(self): """ defined state test cases. @@ -1012,126 +644,18 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin): "virt.vm_state": MagicMock(return_value={"myvm": "stopped"}), "virt.start": MagicMock(return_value=0), "virt.update": MagicMock(return_value={"definition": False}), - "virt.list_domains": MagicMock(return_value=["myvm"]), - }, - ): - ret.update( - { - "changes": {"myvm": {"started": True}}, - "comment": "Domain myvm started", - } - ) - self.assertDictEqual(virt.running("myvm"), ret) - - # Test defining and starting a guest the old way - init_mock = MagicMock(return_value=True) - start_mock = MagicMock(return_value=0) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.vm_state": MagicMock(return_value={"myvm": "stopped"}), - "virt.init": init_mock, - "virt.start": start_mock, - "virt.list_domains": MagicMock(return_value=[]), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True, "started": True}}, - "comment": "Domain myvm defined and started", - } - ) - self.assertDictEqual( - virt.running("myvm", cpu=2, mem=2048, image="/path/to/img.qcow2"), - ret, - ) - init_mock.assert_called_with( - "myvm", - cpu=2, - mem=2048, - os_type=None, - arch=None, - boot=None, - disk=None, - disks=[{"name": "system", "image": "/path/to/img.qcow2"}], - nic=None, - interfaces=None, - graphics=None, - hypervisor=None, - start=False, - seed=True, - install=True, - pub_key=None, - priv_key=None, - connection=None, - username=None, - password=None, - ) - start_mock.assert_called_with( - "myvm", connection=None, username=None, password=None - ) - - # Test image parameter with disks with defined image - init_mock = MagicMock(return_value=True) - start_mock = MagicMock(return_value=0) - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.vm_state": MagicMock(return_value={"myvm": "stopped"}), - "virt.init": init_mock, - "virt.start": start_mock, - "virt.list_domains": MagicMock(return_value=[]), - }, - ): - ret.update( - { - "changes": {"myvm": {"definition": True, "started": True}}, - "comment": "Domain myvm defined and started", - } - ) - disks = [ - { - "name": "system", - "size": 8192, - "overlay_image": True, - "pool": "default", - "image": "/path/to/image.qcow2", - }, - {"name": "data", "size": 16834}, - ] - self.assertDictEqual( - virt.running( - "myvm", cpu=2, mem=2048, disks=disks, image="/path/to/img.qcow2" - ), - ret, - ) - init_mock.assert_called_with( - "myvm", - cpu=2, - mem=2048, - os_type=None, - arch=None, - boot=None, - disk=None, - disks=disks, - nic=None, - interfaces=None, - graphics=None, - hypervisor=None, - start=False, - seed=True, - install=True, - pub_key=None, - priv_key=None, - connection=None, - username=None, - password=None, - ) - start_mock.assert_called_with( - "myvm", connection=None, username=None, password=None + "virt.list_domains": MagicMock(return_value=["myvm"]), + }, + ): + ret.update( + { + "changes": {"myvm": {"started": True}}, + "comment": "Domain myvm started", + } ) + self.assertDictEqual(virt.running("myvm"), ret) - # Test image parameter with disks without defined image + # Test defining and starting a guest the old way init_mock = MagicMock(return_value=True) start_mock = MagicMock(return_value=0) with patch.dict( @@ -1149,18 +673,12 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin): "comment": "Domain myvm defined and started", } ) - disks = [ - { - "name": "system", - "size": 8192, - "overlay_image": True, - "pool": "default", - }, - {"name": "data", "size": 16834}, - ] self.assertDictEqual( virt.running( - "myvm", cpu=2, mem=2048, disks=disks, image="/path/to/img.qcow2" + "myvm", + cpu=2, + mem=2048, + disks=[{"name": "system", "image": "/path/to/img.qcow2"}], ), ret, ) @@ -1172,16 +690,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin): arch=None, boot=None, disk=None, - disks=[ - { - "name": "system", - "size": 8192, - "overlay_image": True, - "pool": "default", - "image": "/path/to/img.qcow2", - }, - {"name": "data", "size": 16834}, - ], + disks=[{"name": "system", "image": "/path/to/img.qcow2"}], nic=None, interfaces=None, graphics=None, @@ -2100,199 +1609,6 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin): virt.network_defined("mynet", "br2", "bridge"), ret ) - def test_network_defined(self): - """ - network_defined state test cases. - """ - ret = {"name": "mynet", "changes": {}, "result": True, "comment": ""} - with patch.dict(virt.__opts__, {"test": False}): - define_mock = MagicMock(return_value=True) - # Non-existing network case - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.network_info": MagicMock( - side_effect=[{}, {"mynet": {"active": False}}] - ), - "virt.network_define": define_mock, - }, - ): - ret.update( - { - "changes": {"mynet": "Network defined"}, - "comment": "Network mynet defined", - } - ) - self.assertDictEqual( - virt.network_defined( - "mynet", - "br2", - "bridge", - vport="openvswitch", - tag=180, - ipv4_config={ - "cidr": "192.168.2.0/24", - "dhcp_ranges": [ - {"start": "192.168.2.10", "end": "192.168.2.25"}, - {"start": "192.168.2.110", "end": "192.168.2.125"}, - ], - }, - ipv6_config={ - "cidr": "2001:db8:ca2:2::1/64", - "dhcp_ranges": [ - { - "start": "2001:db8:ca2:1::10", - "end": "2001:db8:ca2::1f", - }, - ], - }, - autostart=False, - connection="myconnection", - username="user", - password="secret", - ), - ret, - ) - define_mock.assert_called_with( - "mynet", - "br2", - "bridge", - vport="openvswitch", - tag=180, - autostart=False, - start=False, - ipv4_config={ - "cidr": "192.168.2.0/24", - "dhcp_ranges": [ - {"start": "192.168.2.10", "end": "192.168.2.25"}, - {"start": "192.168.2.110", "end": "192.168.2.125"}, - ], - }, - ipv6_config={ - "cidr": "2001:db8:ca2:2::1/64", - "dhcp_ranges": [ - {"start": "2001:db8:ca2:1::10", "end": "2001:db8:ca2::1f"}, - ], - }, - connection="myconnection", - username="user", - password="secret", - ) - - # Case where there is nothing to be done - define_mock.reset_mock() - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.network_info": MagicMock( - return_value={"mynet": {"active": True}} - ), - "virt.network_define": define_mock, - }, - ): - ret.update({"changes": {}, "comment": "Network mynet exists"}) - self.assertDictEqual( - virt.network_defined("mynet", "br2", "bridge"), ret - ) - - # Error case - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.network_info": MagicMock(return_value={}), - "virt.network_define": MagicMock( - side_effect=self.mock_libvirt.libvirtError("Some error") - ), - }, - ): - ret.update({"changes": {}, "comment": "Some error", "result": False}) - self.assertDictEqual( - virt.network_defined("mynet", "br2", "bridge"), ret - ) - - # Test cases with __opt__['test'] set to True - with patch.dict(virt.__opts__, {"test": True}): - ret.update({"result": None}) - - # Non-existing network case - define_mock.reset_mock() - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.network_info": MagicMock(return_value={}), - "virt.network_define": define_mock, - }, - ): - ret.update( - { - "changes": {"mynet": "Network defined"}, - "comment": "Network mynet defined", - } - ) - self.assertDictEqual( - virt.network_defined( - "mynet", - "br2", - "bridge", - vport="openvswitch", - tag=180, - ipv4_config={ - "cidr": "192.168.2.0/24", - "dhcp_ranges": [ - {"start": "192.168.2.10", "end": "192.168.2.25"}, - {"start": "192.168.2.110", "end": "192.168.2.125"}, - ], - }, - ipv6_config={ - "cidr": "2001:db8:ca2:2::1/64", - "dhcp_ranges": [ - { - "start": "2001:db8:ca2:1::10", - "end": "2001:db8:ca2::1f", - }, - ], - }, - autostart=False, - connection="myconnection", - username="user", - password="secret", - ), - ret, - ) - define_mock.assert_not_called() - - # Case where there is nothing to be done - define_mock.reset_mock() - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.network_info": MagicMock( - return_value={"mynet": {"active": True}} - ), - "virt.network_define": define_mock, - }, - ): - ret.update( - {"changes": {}, "comment": "Network mynet exists", "result": True} - ) - self.assertDictEqual( - virt.network_defined("mynet", "br2", "bridge"), ret - ) - - # Error case - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.network_info": MagicMock( - side_effect=self.mock_libvirt.libvirtError("Some error") - ) - }, - ): - ret.update({"changes": {}, "comment": "Some error", "result": False}) - self.assertDictEqual( - virt.network_defined("mynet", "br2", "bridge"), ret - ) - def test_network_running(self): """ network_running state test cases. @@ -3303,45 +2619,6 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin): ret, ) - # test case with test=True and pool to be defined - for mock in mocks: - mocks[mock].reset_mock() - with patch.dict( - virt.__salt__, - { # pylint: disable=no-member - "virt.pool_info": MagicMock(return_value={}), - }, - ): - ret.update( - { - "changes": { - "mypool": "Pool defined, marked for autostart, started" - }, - "comment": "Pool mypool defined, marked for autostart, started", - "result": None, - } - ) - self.assertDictEqual( - virt.pool_running( - "mypool", - ptype="logical", - target="/dev/base", - permissions={ - "mode": "0770", - "owner": 1000, - "group": 100, - "label": "seclabel", - }, - source={"devices": [{"path": "/dev/sda"}]}, - transient=True, - autostart=True, - connection="myconnection", - username="user", - password="secret", - ), - ret, - ) - def test_pool_deleted(self): """ Test the pool_deleted state -- 2.29.2
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Code of Conduct
Contact
Support
@OBShq
Terms
openSUSE Build Service is sponsored by
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor
