Support VPC public gateway rate throttling, built on NIC/network rate persistence and precedence fixes by sudo87 · Pull Request #13325 · apache/cloudstack

sudo87 · 2026-06-03T09:56:15Z

Description

Adds an operator-configurable data transfer rate for a VPC's public/internet-facing gateway, independent of the per-tier rates that network offerings already control.

Precedence:

VPC offering  rate >  vpc.public.network.throttling.rate (default unlimited)

Also persists the effective network rate per NIC and Network and exposes the effective network rate (bandwidth throttling) configured for NICs and guest networks in the API responses and UI.

Also includes the 4.23.0.0 → 24.0.0 version upgrade path from the PR #14033, so our migration targets a version that'll actually exist. Will need reconciling once that PR lands.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Verified end-to-end in a lab: API responses, the actual libvirt bandwidth configuration on the router, and measured live throughput across a multi-tier VPC confirming tiers correctly share one capped public-gateway.

How did you try to break this feature and the system with this change?

codecov · 2026-06-03T10:30:01Z

Codecov Report

❌ Patch coverage is 8.09859% with 261 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.91%. Comparing base (602d9ec) to head (c9b2ce2).

Files with missing lines	Patch %	Lines
...in/java/com/cloud/upgrade/NetworkRateBackfill.java	0.00%	148 Missing ⚠️
...ain/java/com/cloud/network/vpc/VpcManagerImpl.java	41.66%	11 Missing and 3 partials ⚠️
.../cloud/configuration/ConfigurationManagerImpl.java	0.00%	12 Missing ⚠️
...java/com/cloud/upgrade/dao/Upgrade42300to2400.java	0.00%	10 Missing ⚠️
...src/main/java/com/cloud/api/ApiResponseHelper.java	0.00%	9 Missing ⚠️
...tack/engine/orchestration/NetworkOrchestrator.java	12.50%	7 Missing ⚠️
...pache/cloudstack/api/response/NetworkResponse.java	0.00%	6 Missing ⚠️
...rg/apache/cloudstack/api/response/NicResponse.java	0.00%	6 Missing ⚠️
...main/java/com/cloud/network/vpc/VpcOfferingVO.java	0.00%	6 Missing ⚠️
...ngine/schema/src/main/java/com/cloud/vm/NicVO.java	0.00%	6 Missing ⚠️
... and 14 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #13325      +/-   ##
============================================
- Coverage     19.91%   19.91%   -0.01%     
- Complexity    20200    20221      +21     
============================================
  Files          6373     6374       +1     
  Lines        577230   577508     +278     
  Branches      70696    70748      +52     
============================================
+ Hits         114974   115029      +55     
- Misses       449690   449904     +214     
- Partials      12566    12575       +9

Flag	Coverage Δ
uitests	`3.70% <ø> (-0.01%)`	⬇️
unittests	`21.18% <8.09%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…e offering

github-actions · 2026-06-08T07:10:22Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

# Conflicts: # server/src/main/java/com/cloud/vm/UserVmManagerImpl.java # ui/src/config/section/network.js

- Add network_rate column to nics table (schema-42300to42400.sql) - Add DB upgrade path: Upgrade42300to42400 registered in DatabaseUpgradeChecker - Add network_rate field and getter/setter to NicVO - Set network_rate on NicVO in NetworkOrchestrator.allocateNic() where rate is already computed, eliminating secondary per-NIC update calls - Add getNetworkRate() to Nic interface so ApiResponseHelper.createNicResponse can call result.getNetworkRate() without casting or extra DB queries - Add nic_network_rate to user_vm_view and UserVmJoinVO so listVirtualMachines reads rate from the join without extra per-NIC findNicById calls - Update UserVmJoinDaoImpl to use uvo.getNicNetworkRate() directly - Expose network_rate in NicResponse as Integer (null = unlimited) - Refresh NIC rates on VM start via refreshNicNetworkRates in UserVmManagerImpl

sudo87 · 2026-09-04T11:27:37Z

@blueorangutan package

blueorangutan · 2026-09-04T11:28:05Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-09-04T12:15:36Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19103

github-actions · 2026-09-07T09:45:12Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

sudo87 · 2026-09-07T16:55:53Z

@blueorangutan package

blueorangutan · 2026-09-07T16:58:04Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-09-07T17:47:49Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19139

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Refactor versioning logic to handle four-component metadata separately. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…0 versioning cutover Cherry-picked the upstream 4.23.0.0 -> 24.0.0 version-cutover upgrade path (management server versions dropping the leading "4." from 24.0.0 onwards) so our own migration targets the real next version. Our previous Upgrade42300to42400 registered itself against "4.24.0.0", which no longer exists once the cutover lands - the project's actual next version is 24.0.0, matching pom.xml's 24.0.0-SNAPSHOT. Folded the public_nw_rate/network_rate column additions and the NetworkRateBackfill data migration into the new Upgrade42300to2400 class and its schema-42300to2400.sql, and removed the now-redundant Upgrade42300to42400/schema-42300to42400.sql.

sudo87 · 2026-09-18T08:19:19Z

@blueorangutan package

blueorangutan · 2026-09-18T08:20:05Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-09-18T09:10:44Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19281

github-actions · 2026-09-18T10:28:22Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

# Conflicts: # engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade42300to2400.java # engine/schema/src/main/resources/META-INF/db/schema-42300to2400.sql

Found by actually running the 4.23.0.0 -> 24.0.0 upgrade path via RPM against a real DB, not just unit tests: - findRouterGuestNetworkRate() called NicDao.listByVmId(), but this class's DAOs are constructed outside Spring during upgrade bootstrap, and that query relies on a SearchBuilder that isn't initialized in that context, throwing an NPE that was silently caught. Every VPC/ isolated-network router's public NIC failed to backfill as a result. Replaced with a raw SQL query, matching this class's existing pattern for DAOs that aren't safe to use directly here. - backfillNicNetworkRates() only persisted a computed rate when it was greater than zero, silently skipping the legitimate "-1 (unlimited)" case - affecting ConsoleProxy/SecondaryStorageVm NICs and any NIC whose legacy rate resolved to unlimited. Left network_rate NULL instead of -1, which the API renders as blank instead of "Unlimited" until the NIC is reallocated.

nics.network_rate is an unsigned int column, so it cannot store -1; NULL is the established convention for "unlimited" there, exactly matching how the live precedence code (NetworkOrchestrator.allocateNic()) already stores it. Changing the backfill's guard to write -1 caused a real "Data truncation: Out of range value" failure, confirmed via a second real upgrade run. The original rate > 0 check (skip writing on unlimited, leaving NULL) was correct; only the router public-NIC NPE fix from the previous commit was a genuine bug.

nics.network_rate was unsigned, so the only way to represent "unlimited" was NULL, even though every response-building call site already normalizes null/non-positive to -1 for API output - meaning NULL and -1 were already behaviorally equivalent, just inconsistent at the storage layer. NetworkModelImpl.getNetworkRate() never actually returns null (always a positive rate or -1), so the null-coalescing in NetworkOrchestrator was dead code once the column can hold -1 directly. Make the column signed so it can store -1 like every other "resolved" rate value in the codebase, simplify both NetworkOrchestrator call sites to store the value as-is, and let the backfill persist -1 for NICs whose legacy rate is unlimited instead of leaving them NULL.

sudo87 · 2026-09-18T12:18:11Z

@blueorangutan package

blueorangutan · 2026-09-18T12:20:05Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-09-18T13:10:58Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19283

boring-cyborg Bot added component:api component:networking component:orchestration component:UI labels Jun 3, 2026

sudo87 changed the title ~~persist and expose effective network rate for NIC, Network and compute offering~~ Persist and expose effective network rate for NIC, Network and compute offering Jun 3, 2026

sudo87 marked this pull request as draft June 3, 2026 09:56

bernardodemarco requested a review from hsato03 June 3, 2026 12:23

persist and expose effective network rate for NIC, Network and comput…

498b69b

…e offering

github-actions Bot added the status:has-conflicts label Jun 8, 2026

weizhouapache added this to the 4.24.0 milestone Jun 29, 2026

DaanHoogland added this to CloudStack Testing Aug 31, 2026

github-project-automation Bot moved this to Backlog in CloudStack Testing Aug 31, 2026

DaanHoogland moved this from Backlog to conflict/waiting for author in CloudStack Testing Aug 31, 2026

Merge branch 'main' into networkThrottling

c8ac2a0

# Conflicts: # server/src/main/java/com/cloud/vm/UserVmManagerImpl.java # ui/src/config/section/network.js

sudo87 force-pushed the networkThrottling branch from 35bcb26 to c8ac2a0 Compare September 4, 2026 06:14

github-actions Bot removed the status:has-conflicts label Sep 4, 2026

github-actions Bot added the status:has-conflicts label Sep 7, 2026

Merge branch 'main' into networkThrottling

fb7cce0

github-actions Bot removed the status:has-conflicts label Sep 7, 2026

fix ui build due to merge conflict

b684b88

shwstppr and others added 15 commits September 18, 2026 12:42

fix CS version

18c74d3

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix upgrade unit tests for cutover

29b9b76

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix imports

ec174a1

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix template version

673fa86

continue using 3 part version for sys template

20f686c

fix

8720eb3

fix more

d4c3549

use security version after cutover

0f4c5ef

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

improvements and add tests

e1b7a2f

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix more scripts

d6198c6

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

address remaining comments

5f8b0dc

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

more changes

dec9da9

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Update versioning logic in export-templates.sh

1fbbe0f

Refactor versioning logic to handle four-component metadata separately. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Fix export-templates.sh syntax

29415b3

boring-cyborg Bot added the component:build label Sep 18, 2026

sudo87 changed the title ~~Persist and expose effective network rate for NIC, Network and compute offering~~ Support VPC public gateway rate throttling, built on NIC/network rate persistence and precedence fixes Sep 18, 2026

github-actions Bot added the status:has-conflicts label Sep 18, 2026

Merge remote-tracking branch 'apache/main' into networkThrottling

d5428f1

# Conflicts: # engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade42300to2400.java # engine/schema/src/main/resources/META-INF/db/schema-42300to2400.sql

github-actions Bot removed the status:has-conflicts label Sep 18, 2026

sudo87 added 3 commits September 18, 2026 17:03

Conversation

sudo87 commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Jun 8, 2026

Uh oh!

sudo87 commented Sep 4, 2026

Uh oh!

blueorangutan commented Sep 4, 2026

Uh oh!

blueorangutan commented Sep 4, 2026

Uh oh!

github-actions Bot commented Sep 7, 2026

Uh oh!

sudo87 commented Sep 7, 2026

Uh oh!

blueorangutan commented Sep 7, 2026

Uh oh!

blueorangutan commented Sep 7, 2026

Uh oh!

sudo87 commented Sep 18, 2026

Uh oh!

blueorangutan commented Sep 18, 2026

Uh oh!

blueorangutan commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

sudo87 commented Sep 18, 2026

Uh oh!

blueorangutan commented Sep 18, 2026

Uh oh!

blueorangutan commented Sep 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sudo87 commented Jun 3, 2026 •

edited

Loading

codecov Bot commented Jun 3, 2026 •

edited

Loading