Skip to content

compute: normalize cross-HV BDM device names - #653

Open
anokfireball wants to merge 1 commit into
stable/2023.2-m3from
feat/kvmotion-custom-device-path-handling
Open

compute: normalize cross-HV BDM device names#653
anokfireball wants to merge 1 commit into
stable/2023.2-m3from
feat/kvmotion-custom-device-path-handling

Conversation

@anokfireball

Copy link
Copy Markdown
Member

Customers can set their own BDM device paths. VMware does not pass these paths to the guest as disk targets. It uses them to keep a stable volume attachment order, while the guest still sees normal SCSI devices and /dev/sdX device names in guest.

This is not compatible with the current libvirt/KVM path. Nova passes the BDM device name to libvirt as the disk target, and libvirt expects a guest disk target name for the selected bus.

During VMware-to-KVM resize, journal the original root and BDM names. Then assign /dev/vdX target device names following the VMware order: root first, then remaining volumes by original device name.

On revert, restore the journaled root name, BDM names, and disk buses exactly.

Change-Id: Id130a3ef18be29bfb8bab3681c75554b23862ba5

Customers can set their own BDM device paths. VMware does not pass these
paths to the guest as disk targets. It uses them to keep a stable volume
attachment order, while the guest still sees normal SCSI devices and
/dev/sdX device names in guest.

This is not compatible with the current libvirt/KVM path. Nova passes
the BDM device name to libvirt as the disk target, and libvirt expects a
guest disk target name for the selected bus.

During VMware-to-KVM resize, journal the original root and BDM names.
Then assign /dev/vdX target device names following the VMware order:
root first, then remaining volumes by original device name.

On revert, restore the journaled root name, BDM names, and disk buses
exactly.

Change-Id: Id130a3ef18be29bfb8bab3681c75554b23862ba5
Comment thread nova/compute/manager.py

def _journal_cross_hv_bdms(self, instance, bdms):
sysmeta = instance.system_metadata
root_name_key = 'cross_hv_orig_root_device_name'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be a constant instead of the same string 3 times but probably won't ever change.

Comment thread nova/compute/manager.py
Comment on lines +6280 to +6286
def original_bdm_device_name(bdm):
bdm_name_key = (
'cross_hv_orig_bdm_device_name_%s' % bdm.volume_id)
if bdm_name_key in sysmeta:
return self._cross_hv_restore_journal_value(
sysmeta[bdm_name_key])
return bdm.device_name

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could also be a staticmethod _original_bdm_device_name(sysmeta, bdm) for one less copy.

Comment thread nova/compute/manager.py
Comment on lines +6347 to +6349
return (not is_root_bdm(bdm),
original_bdm_device_name(bdm) or '',
bdm.volume_id)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So root comes first because it's the only False in the tuple. Second would be any bdm without an original device name because they'd evaluate to ''. Is that the correct order of VMWare?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, true. maybe it should be or 'zzzzzzz' or something? this is just for ordering, so the string itself wouldn't matter.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a genuine question I don't know the answer to, this might be the actual way how VMWare orders it.

self.assertEqual('virtio', root_bdm.disk_bus)
self.assertEqual('virtio', data_bdm.disk_bus)

def test_sanitize_cross_hv_bdms_forward_keeps_existing_journal(self):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of interest, when would a journal exist already? On a VM that was resized and reverted?

@grandchild grandchild left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just nits

Comment thread nova/compute/manager.py
if root_device_name != instance.root_device_name:
instance.root_device_name = root_device_name
def _cross_hv_journal_value(value):
return value if value is not None else '__NONE__'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really expect empty string or 0, do we?

Suggested change
return value if value is not None else '__NONE__'
return value or '__NONE__'

... and then one could almost inline the function again.

Comment thread nova/compute/manager.py
Comment on lines +6325 to +6331
def original_bdm_device_name(bdm):
bdm_name_key = (
'cross_hv_orig_bdm_device_name_%s' % bdm.volume_id)
if bdm_name_key in sysmeta:
return self._cross_hv_restore_journal_value(
sysmeta[bdm_name_key])
return bdm.device_name

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a duplicate with the one in _journal_cross_hv_bdms() -- i'd pull it out into a static member function...

Comment thread nova/compute/manager.py
Comment on lines +6347 to +6349
return (not is_root_bdm(bdm),
original_bdm_device_name(bdm) or '',
bdm.volume_id)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, true. maybe it should be or 'zzzzzzz' or something? this is just for ordering, so the string itself wouldn't matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants