test: fix smoke test failures on xen/vmware by weizhouapache · Pull Request #13628 · apache/cloudstack

weizhouapache · 2026-07-16T11:34:21Z

Description

This PR fixes some smoke test failures on vmware or xenserver/xcpng

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

weizhouapache · 2026-07-16T11:34:36Z

@blueorangutan package

blueorangutan · 2026-07-16T11:36:05Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Copilot

Pull request overview

This PR targets integration smoke test stability across hypervisors (notably XenServer/XCP-ng and VMware) by skipping KVM-specific checks/tests when not running on KVM, and by adjusting a few timing/host prerequisite assumptions that can cause intermittent failures.

Changes:

Skip KVM-only paths in smoke tests when the detected hypervisor is not KVM.
Add additional host-tool prerequisite checks for a VPC source NAT IP update test.
Add delays around affinity-group migration steps and make the events-resource test more tolerant around volume attach/detach operations.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
test/integration/smoke/test_nonstrict_affinity_group.py	Adds waits before migration steps in non-strict affinity group tests.
test/integration/smoke/test_network_extension_namespace.py	Skips KVM prerequisite checks on non-KVM hypervisors; expands required host tools for one test.
test/integration/smoke/test_kms_lifecycle.py	Skips a VM root-disk encryption test on non-KVM hypervisors.
test/integration/smoke/test_events_resource.py	Wraps volume attach/detach operations to avoid hard failures in some environments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-07-16T11:43:10Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.71%. Comparing base (5812290) to head (f947c85).

❗ There is a different number of reports uploaded between BASE (5812290) and HEAD (f947c85). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (5812290) HEAD (f947c85)

unittests 1 0

Additional details and impacted files

@@              Coverage Diff              @@
##               main   #13628       +/-   ##
=============================================
- Coverage     19.90%    3.71%   -16.20%     
=============================================
  Files          6372      487     -5885     
  Lines        577180    41992   -535188     
  Branches      70693     7942    -62751     
=============================================
- Hits         114869     1558   -113311     
+ Misses       449755    40208   -409547     
+ Partials      12556      226    -12330

Flag	Coverage Δ
uitests	`3.71% <ø> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2026-07-16T12:24:11Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18593

weizhouapache · 2026-07-16T12:35:32Z

@blueorangutan test ubuntu26 xcpng82 keepEnv

blueorangutan · 2026-07-16T12:36:03Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + xcpng82) has been kicked to run smoke tests

weizhouapache · 2026-07-16T12:37:19Z

@blueorangutan test ubuntu26 vmware-80u3e keepEnv

blueorangutan · 2026-07-16T12:38:05Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + vmware-80u3e) has been kicked to run smoke tests

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

weizhouapache · 2026-07-17T11:34:58Z

@blueorangutan package

blueorangutan · 2026-07-17T11:36:04Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-07-17T12:24:46Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18602

weizhouapache · 2026-07-17T12:45:07Z

@blueorangutan test ubuntu26 xcpng82

blueorangutan · 2026-07-17T12:46:03Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + xcpng82) has been kicked to run smoke tests

weizhouapache · 2026-07-17T12:46:57Z

@blueorangutan test ubuntu26 vmware-80u3e

blueorangutan · 2026-07-17T12:48:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + vmware-80u3e) has been kicked to run smoke tests

blueorangutan · 2026-07-17T13:46:10Z

[SF] Trillian Build Failed (tid-7)

weizhouapache · 2026-07-17T15:36:37Z

@blueorangutan test ubuntu26 vmware-80u3

blueorangutan · 2026-07-17T15:38:03Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + vmware-80u3) has been kicked to run smoke tests

blueorangutan · 2026-07-17T23:16:50Z

[SF] Trillian Build Failed (tid-9)

…e.py

…l.py

weizhouapache · 2026-09-17T14:41:47Z

@blueorangutan package

blueorangutan · 2026-09-17T14:44:08Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Copilot

🔵 Needs a closer look

A class setup path in test_usage.py can still crash with an IndexError/TypeError by indexing the first element of a possibly empty/None PublicIPAddress.list() result.

Review details

Suppressed comments (3)

test/integration/smoke/test_usage.py:1756

PublicIPAddress.list(...) can return None or an empty list; indexing src_nat_list[0] will then raise TypeError/IndexError and fail class setup. Add an explicit guard (and a clear failure message) before selecting the first element.

            issourcenat=True
        )
        cls.public_ip = src_nat_list[0]

test/integration/smoke/test_network_extension_namespace.py:863

skipTest() already raises unittest.SkipTest; using raise self.skipTest(...) is redundant and can be confusing (and would become a TypeError if skipTest were ever refactored to return an exception). Call self.skipTest(...) directly.

        if self.hv.lower() != 'kvm':
            raise self.skipTest("Skipping test case for non-kvm hypervisor")

test/integration/smoke/test_kms_lifecycle.py:430

skipTest() already raises unittest.SkipTest; using raise self.skipTest(...) is redundant and can be confusing. Call self.skipTest(...) directly.

        if self.hypervisor.lower() != 'kvm':
            raise self.skipTest("Skipping test case for non-kvm hypervisor")

Files reviewed: 11/11 changed files
Comments generated: 0 new
Review effort level: Lite

blueorangutan · 2026-09-17T15:33:56Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19272

weizhouapache · 2026-09-17T16:59:59Z

@blueorangutan test matrix

blueorangutan · 2026-09-17T17:02:03Z

@weizhouapache a [SL] Trillian-Jenkins matrix job (ol8 mgmt + kvm-ol8, ubuntu22 mgmt + kvm-ubuntu22, ol8 mgmt + vmware-70u3, ol9 mgmt + xcpng82) has been kicked to run smoke tests

blueorangutan · 2026-09-17T17:53:34Z

[SF] Trillian Build Failed (tid-17017)

weizhouapache · 2026-09-17T18:05:24Z

@blueorangutan test ol9 xcpng82

blueorangutan · 2026-09-17T18:06:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ol9 mgmt + xcpng82) has been kicked to run smoke tests

blueorangutan · 2026-09-18T08:48:14Z

[SF] Trillian test result (tid-17014)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 52932 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t17014-kvm-ol8.zip
Smoke tests completed. 156 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2026-09-18T11:28:38Z

[SF] Trillian test result (tid-17015)
Environment: kvm-ubuntu22 (x2), zone: Advanced Networking with Mgmt server u22
Total time taken: 63458 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t17015-kvm-ubuntu22.zip
Smoke tests completed. 156 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2026-09-18T20:34:50Z

[SF] Trillian test result (tid-17018)
Environment: xcpng82 (x2), zone: Advanced Networking with Mgmt server ol9
Total time taken: 92929 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t17018-xcpng82.zip
Smoke tests completed. 155 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_11_destroy_vm_and_volumes	`Error`	34.16	test_vm_life_cycle.py

blueorangutan · 2026-09-19T06:48:10Z

[SF] Trillian test result (tid-17016)
Environment: vmware-70u3 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 131950 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t17016-vmware-70u3.zip
Smoke tests completed. 153 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_deploy_vm_on_specific_host	`Error`	23.81	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	`Error`	3606.01	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	`Error`	24.85	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	`Error`	3603.03	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	`Error`	4.38	test_vm_deployment_planner.py
test_09_expunge_vm	`Failure`	427.87	test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success	`Error`	3608.56	test_vm_strict_host_tags.py
test_02_migrate_vm_strict_tags_failure	`Error`	20.28	test_vm_strict_host_tags.py
test_01_restore_vm_strict_tags_success	`Error`	3605.24	test_vm_strict_host_tags.py
test_02_restore_vm_strict_tags_failure	`Error`	4.91	test_vm_strict_host_tags.py
test_01_scale_vm_strict_tags_success	`Error`	20.09	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	`Error`	3608.11	test_vm_strict_host_tags.py
test_01_deploy_vm_on_specific_host_without_strict_tags	`Error`	26.24	test_vm_strict_host_tags.py
test_02_deploy_vm_on_any_host_without_strict_tags	`Error`	7207.20	test_vm_strict_host_tags.py
test_03_deploy_vm_on_specific_host_with_strict_tags_success	`Error`	3606.29	test_vm_strict_host_tags.py
test_04_deploy_vm_on_any_host_with_strict_tags_success	`Error`	5.81	test_vm_strict_host_tags.py

Copilot AI lite review requested due to automatic review settings July 16, 2026 11:34

boring-cyborg Bot added component:integration-test Python Warning... Python code Ahead! labels Jul 16, 2026

weizhouapache added this to the 4.23.0 milestone Jul 16, 2026

Copilot started reviewing on behalf of weizhouapache July 16, 2026 11:34 View session

weizhouapache removed this from the 4.23.0 milestone Jul 16, 2026

weizhouapache added this to the 4.23.0 milestone Jul 16, 2026

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread test/integration/smoke/test_nonstrict_affinity_group.py

Comment thread test/integration/smoke/test_nonstrict_affinity_group.py

Comment thread test/integration/smoke/test_events_resource.py Outdated

Copilot AI review requested due to automatic review settings July 17, 2026 09:41

Copilot started reviewing on behalf of weizhouapache July 17, 2026 09:42 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread test/integration/smoke/test_network_extension_namespace.py

Comment thread test/integration/smoke/test_kms_lifecycle.py

Comment thread test/integration/smoke/test_events_resource.py

github-project-automation Bot moved this to Backlog in CloudStack Testing Aug 31, 2026

DaanHoogland moved this from Backlog to In review in CloudStack Testing Sep 1, 2026

weizhouapache commented Sep 1, 2026

View reviewed changes

Comment thread ...datamotion/src/main/java/org/apache/cloudstack/storage/motion/AncientDataMotionStrategy.java

weizhouapache modified the milestones: 4.23.0, 24.0.0 Sep 1, 2026

weizhouapache added 9 commits September 17, 2026 16:40

test: fix smoke test failures on xen/vmware

bdbe57c

test: sleep 10 seconds before attaching volume in test_events_resourc…

a02e145

…e.py

test: add more time.sleep(10)

89233ab

test: add more time.sleep(10) in test_vm_strict_host_tags.py

f75bf87

test: fix test_usage.py

7ff6d1f

test: move time.sleep(10) in test_host_maintenance.py

ae8f612

VMware: fix import DC

992bc90

test: retry domain deletion in tearDown of test_deploy_vms_in_paralle…

e4e403e

…l.py

test: fix tests failures caused by others in drs and host maintenance

f947c85

Copilot AI review requested due to automatic review settings September 17, 2026 14:40

weizhouapache force-pushed the 4.23-fix-smoke-test-failures branch from 2addfa1 to f947c85 Compare September 17, 2026 14:40

Copilot started reviewing on behalf of weizhouapache September 17, 2026 14:41 View session

Copilot AI reviewed Sep 17, 2026

View reviewed changes

Conversation

weizhouapache commented Jul 16, 2026

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

weizhouapache commented Jul 16, 2026

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

weizhouapache commented Jul 16, 2026

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

weizhouapache commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

Uh oh!

weizhouapache commented Sep 17, 2026

Uh oh!

blueorangutan commented Sep 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (3)

Uh oh!

blueorangutan commented Sep 17, 2026

Uh oh!

weizhouapache commented Sep 17, 2026

codecov Bot commented Jul 16, 2026 •

edited

Loading

weizhouapache commented Jul 16, 2026 •

edited

Loading