[Veeam] externalize restore timeout

[SadiJr]

Description

Using the VMWare hypervisor with Veeam integration enabled, ACS uses a hard-coded timeout of 600 seconds (10 minutes) to monitor restore process status in Veeam. If the restore does not finish in this time, ACS throws an exception; however, the restore process continues in Veeam. This PR aims to externalize a global configuration to allow operators to define the timeout value in Veeam restore backups process. Thus, making it adaptable to different needs.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

How Has This Been Tested?

It was tested in a local lab:

1. I created a new huge VM and attached this VM to a Backup Offering;
2. I performed a manual backup;
3. I tried to restore this VM using the backup;
4. Before, an exception was thrown, but the restore backup process keep running in Veeam;
5. Now, with a bigger timeout value, the restore process finish with success.

[acs-robot]

Found UI changes, kicking a new UI QA build
@blueorangutan ui

[blueorangutan]

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

[blueorangutan]

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6320 (SL-JID-1475)

[acs-robot]

PR Coverage Report

CLASS	INSTRUCTION MISSED	INSTRUCTION COVERED	BRANCH MISSED	BRANCH COVERED	LINE MISSED	LINE COVERED
Network	554	0	42	0	107	0
Volume	109	0	2	0	44	0
VirtualMachineGuru	78	0	4	0	15	0
NetworkOrchestrationService	0	101	0	0	0	10
AlertManager	79	0	0	0	7	0
ConnectedAgentAttache	149	0	20	0	40	0
VirtualMachineManagerImpl	15659	0	1500	0	3079	0
NetworkOrchestrator	9910	0	1224	0	1945	0
CapacityVO	261	0	0	0	80	0
DataCenterVnetVO	58	0	0	0	24	0
VlanVO	186	0	2	0	72	0
VlanDaoImpl	1307	0	54	0	218	0
EventVO	140	0	0	0	58	0
HostDaoImpl	4983	0	180	0	803	0
AccountGuestVlanMapVO	46	0	0	0	19	0
NetworkDetailsDaoImpl	30	0	4	0	5	0
NetworkVO	396	159	19	1	140	44
VpcOfferingDaoImpl	137	0	0	0	25	0
VpcOfferingDetailsDaoImpl	173	0	10	0	31	0
NetworkOfferingVO	414	31	0	0	126	12
NetworkOfferingDaoImpl	691	0	22	0	114	0
VolumeVO	529	133	4	0	179	39
SystemVmTemplateRegistration	1773	0	86	0	376	0
Upgrade41520to41600	262	17	16	0	62	5
Upgrade41610to41700	159	7	6	0	43	2
DomainRouterVO	116	50	0	0	41	14
ConsoleProxyDaoImpl	635	0	20	0	144	0
DomainRouterDaoImpl	1600	0	22	0	228	0
NicDaoImpl	1392	0	12	0	193	0
SnapshotObject	748	30	64	0	151	11
VolumeObject	773	665	77	39	178	117
CloudStackContextLoaderListener	77	0	2	0	21	0
VeeamBackupProvider	625	0	44	0	105	0
VeeamClient	1650	339	122	22	303	70
LibvirtComputingResource	8888	1988	1087	147	1983	451
LibvirtStartCommandWrapper	75	295	16	22	15	72
MockNetworkManagerImpl	510	0	24	0	88	0
SimulatorManagerImpl	1394	0	214	0	309	0
VmwareResource	20395	0	2274	0	4397	0
VmwareStorageProcessor	9892	9	940	0	2122	2
CitrixResourceBase	14658	557	1456	34	3169	122
CitrixCheckSshCommandWrapper	15	51	3	3	3	14
CitrixNetworkElementCommandWrapper	0	14	0	0	0	4
CitrixRebootRouterCommandWrapper	28	25	3	1	5	7
CitrixStartCommandWrapper	690	62	81	1	116	16
KubernetesClusterManagerImpl	4724	0	480	0	760	0
KubernetesClusterActionWorker	1540	0	114	0	281	0
KubernetesClusterResourceModifierActionWorker	1843	0	140	0	325	0
CreateKubernetesClusterCmd	215	0	18	0	61	0
MetricsServiceImpl	2130	0	134	0	446	0
ClusterMetricsResponse	523	0	132	0	60	0
VmMetricsResponse	142	0	22	0	25	0
VolumeMetricsResponse	69	0	8	0	10	0
ZoneMetricsResponse	501	0	126	0	56	0
BigSwitchBcfGuestNetworkGuru	662	0	62	0	145	0
CreateServiceInstanceCmd	201	0	16	0	45	0
ContrailManagerImpl	2218	0	234	0	497	0
StopNetScalerVMCmd	114	0	12	0	26	0
OvsGuestNetworkGuru	409	0	48	0	90	0
VxlanGuestNetworkGuru	89	180	11	21	16	39
LinkAccountToLdapCmd	77	105	6	4	18	23
ListAndSwitchSAMLAccountCmd	182	205	30	22	38	34
DomainChecker	1206	0	300	0	238	0
AlertManagerImpl	2133	0	183	0	418	0
ApiAsyncJobDispatcher	212	0	12	0	53	0
ApiDBUtils	2374	0	210	0	592	0
ApiDispatcher	231	0	42	0	55	0
ApiResponseHelper	12400	0	1292	0	2837	0
ApiServer	2868	112	346	0	672	14
ResponseObjectTypeAdapter	161	8	14	0	37	2
ParamProcessWorker	1050	0	155	0	241	0
QueryManagerImpl	14429	0	1270	0	2439	0
DomainRouterJoinDaoImpl	801	0	80	0	195	0
NetworkOfferingJoinDaoImpl	278	0	12	0	57	0
VolumeJoinDaoImpl	770	0	94	0	171	0
VpcOfferingJoinDaoImpl	172	0	6	0	36	0
AsyncJobJoinVO	86	0	0	0	31	0
DomainRouterJoinVO	237	0	0	0	80	0
EventJoinVO	100	0	0	0	35	0
NetworkOfferingJoinVO	219	0	0	0	79	0
VolumeJoinVO	267	0	0	0	93	0
VpcOfferingJoinVO	120	0	0	0	37	0
ConfigurationManagerImpl	18652	0	3092	0	3639	0
ConsoleProxyManagerImpl	3717	0	423	0	726	0
ActionEventInterceptor	398	19	44	2	90	7
ActionEventUtils	929	0	90	0	198	0
EventJoinDaoImpl	329	0	18	0	58	0
HypervisorGuruBase	669	38	52	0	133	5
LibvirtServerDiscoverer	994	0	116	0	218	0
ExternalFirewallDeviceManagerImpl	1885	0	198	0	381	0
ExternalLoadBalancerDeviceManagerImpl	2591	0	288	0	593	0
IpAddressManagerImpl	4084	0	463	0	814	0
Ipv6AddressManagerImpl	371	0	32	0	70	0
NetworkMigrationManagerImpl	1706	0	90	0	355	0
NetworkModelImpl	6182	0	838	0	1300	0
NetworkServiceImpl	13648	0	1902	0	2613	0
FirewallManagerImpl	2600	0	399	0	466	0
ExternalGuestNetworkGuru	849	0	92	0	158	0
GuestNetworkGuru	702	298	114	34	142	64
PrivateNetworkGuru	394	0	46	0	88	0
PublicNetworkGuru	341	0	42	0	77	0
LoadBalancingRulesManagerImpl	6024	0	666	0	1254	0
CommandSetupHelper	4088	0	300	0	771	0
NetworkHelperImpl	2092	0	278	0	445	0
VirtualNetworkApplianceManagerImpl	7767	0	792	0	1544	0
RulesManagerImpl	4074	0	492	0	790	0
NetworkACLManagerImpl	861	0	118	0	182	0
NetworkACLServiceImpl	2701	0	304	0	520	0
VpcManagerImpl	6936	0	770	0	1321	0
Site2SiteVpnManagerImpl	2298	0	214	0	468	0
ResourceManagerImpl	8524	0	982	0	1659	0
RollingMaintenanceManagerImpl	2041	0	192	0	363	0
ResourceIconManagerImpl	328	0	36	0	70	0
ConfigurationServerImpl	2061	0	176	0	495	0
ManagementServerImpl	12197	0	1076	0	2428	0
StatsCollector	2178	0	124	0	359	0
StorageManagerImpl	8580	0	976	0	1710	0
VolumeApiServiceImpl	10873	0	1502	0	2041	0
SnapshotManagerImpl	4195	0	410	0	753	0
SnapshotSchedulerImpl	871	0	81	0	197	0
ResourceManagerUtilImpl	356	0	24	0	71	0
TemplateManagerImpl	5011	0	696	0	1045	0
AccountManagerImpl	6460	0	906	0	1352	0
UserVmManagerImpl	20798	0	2566	0	3869	0
VMSnapshotManagerImpl	3151	0	296	0	628	0
RoleManagerImpl	737	0	114	0	147	0
AffinityGroupServiceImpl	791	0	112	0	172	0
AnnotationManagerImpl	1401	0	167	0	284	0
BackupManagerImpl	2779	0	224	0	489	0
CAManagerImpl	633	0	80	0	129	0
DirectDownloadManagerImpl	1684	0	186	0	361	0
HAManagerImpl	1391	0	194	0	231	0
BasicNetworkVisitor	535	0	28	0	112	0
OutOfBandManagementServiceImpl	1182	0	130	0	237	0
PowerOperationTask	109	0	0	0	17	0
MockNetworkManagerImpl	510	0	24	0	88	0
SecondaryStorageManagerImpl	3544	149	343	11	629	32
VirtualMachineMO	8953	135	1063	17	1982	31

[yadvr]

Pl see https://sonarcloud.io/summary/new_code?id=apachecloudstack&pullRequest=6320

[yadvr]

Can you merge latest main branch to your PR branch @SadiJr ?

[sonarqubecloud]

SonarCloud Quality Gate failed.   

1 Bug
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

55.0% Coverage
0.0% Duplication

[DaanHoogland]

clgtm

[acs-robot]

Found UI changes, kicking a new UI QA build
@blueorangutan ui

[sureshanaparti]

code LGTM

[sureshanaparti]

@blueorangutan package

[blueorangutan]

@sureshanaparti a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3633

[sureshanaparti]

@blueorangutan test centos7 vmware-67u3

[blueorangutan]

@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

[harikrishna-patnala]

LGTM