Prometheus exporter fix cpu/memory usage labels

[soreana]

Description

This PR fixes the prometheus exporter issue that reports used memory instead of allocated memory.

Closes #7625

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

How Has This Been Tested?

The prometheus exporter output now matches the cmk output. Check out the #7625 for more details:

Memory

root@mgt01:~# curl http://127.0.0.1:9595/metrics | grep cloudstack_host_memory_usage_mibs_total
cloudstack_host_memory_usage_mibs_total{zone="Zone1",hostname="node01",ip="10.1.2.5",overprovisioningfactor="1.0",filter="allocated",dedicated="0",tags=""} 512.00
cloudstack_host_memory_usage_mibs_total{zone="Zone1",hostname="node01",ip="10.1.2.5",overprovisioningfactor="1.0",filter="total",dedicated="0",tags=""} 2907.71
cloudstack_host_memory_usage_mibs_total{zone="Zone1",hostname="node02",ip="10.1.2.6",overprovisioningfactor="1.0",filter="allocated",dedicated="0",tags=""} 1024.00
cloudstack_host_memory_usage_mibs_total{zone="Zone1",hostname="node02",ip="10.1.2.6",overprovisioningfactor="1.0",filter="total",dedicated="0",tags=""} 2907.70
cloudstack_host_memory_usage_mibs_total{zone="Zone1",filter="allocated"} 1536.00
root@mgt01:~# cmk list hostsmetrics filter=memoryallocated,memoryused,memorytotal,name
{
  "count": 2,
  "host": [
    {
      "memoryallocated": 1073741824,
      "memorytotal": 3048947712,
      "memoryused": 1249058816,
      "name": "node02"
    },
    {
      "memoryallocated": 536870912,
      "memorytotal": 3048960000,
      "memoryused": 1262129152,
      "name": "node01"
    }
  ]
}

CPU

root@mgt01:~# curl http://127.0.0.1:9595/metrics | grep cloudstack_host_cpu_usage_mhz_total
cloudstack_host_cpu_usage_mhz_total{zone="Zone1",hostname="node01",ip="10.1.2.5",overprovisioningfactor="1.0",filter="allocated",tags=""} 500.00
cloudstack_host_cpu_usage_mhz_total{zone="Zone1",hostname="node01",ip="10.1.2.5",overprovisioningfactor="1.0",filter="total",tags=""} 4398.00
cloudstack_host_cpu_usage_mhz_total{zone="Zone1",hostname="node02",ip="10.1.2.6",overprovisioningfactor="1.0",filter="allocated",tags=""} 500.00
cloudstack_host_cpu_usage_mhz_total{zone="Zone1",hostname="node02",ip="10.1.2.6",overprovisioningfactor="1.0",filter="total",tags=""} 4400.00
cloudstack_host_cpu_usage_mhz_total{zone="Zone1",filter="allocated"} 1000.00
root@mgt01:~# cmk list hostsmetrics filter=cpuallocatedghz,cpuusedghz,cputotalghz,name
{
  "count": 2,
  "host": [
    {
      "cpuallocatedghz": "0.50 Ghz",
      "cputotalghz": "4.40 Ghz",
      "cpuusedghz": "0.06 Ghz",
      "name": "node02"
    },
    {
      "cpuallocatedghz": "0.50 Ghz",
      "cputotalghz": "4.40 Ghz",
      "cpuusedghz": "0.05 Ghz",
      "name": "node01"
    }
  ]
}

[soreana]

@blueorangutan package

[blueorangutan]

@soreana a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6231

[codecov]

Codecov Report

Merging #7629 (aa53bf2) into 4.18 (fb29608) will increase coverage by 0.03%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##               4.18    #7629      +/-   ##
============================================
+ Coverage     12.98%   13.01%   +0.03%     
- Complexity     8989     9011      +22     
============================================
  Files          2716     2716              
  Lines        256340   256662     +322     
  Branches      39980    40021      +41     
============================================
+ Hits          33291    33410     +119     
- Misses       218883   219066     +183     
- Partials       4166     4186      +20     

Impacted Files	Coverage Δ
...che/cloudstack/metrics/PrometheusExporterImpl.java	0.00% <0.00%> (ø)

... and 15 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[DaanHoogland]

@NuxRo can you have a look at this one (in the light of our discussion on #7591 ) ?

[soreana]

@blueorangutan package

[blueorangutan]

@soreana a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6296

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-7055)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42103 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7629-t7055-kvm-centos7.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_migrate_VM_and_root_volume	Error	77.74	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	Error	53.37	test_vm_life_cycle.py

[DaanHoogland]

@NuxRo can you have a look at this?

[soreana]

@weizhouapache Can you please have a look when you have a time?

[NuxRo]

@soreana @DaanHoogland LGTM after quick testing

[NuxRo]

LGTM

[soreana]

@NuxRo thanks a lot 🙏