vsphere-agents: reduce conflicts

[etungsten]

Issue number:
N/A

Description of changes:

    vpshere-vm: suffix node vm name with uuid
    
    To avoid potential conflicts, suffix node vm names with uuid

    vsphere-agents: delete any conflicting vm/template before starting
    
    Before trying to create a new VM/template with a given name, delete any
    pre-existing vm/template that might pose as a conflict

Testing done:
Ran cargo make test for vmware-k8s-1.28 and I verified it tries removing conflicting VM templates before importing OVA:

$ kubectl --kubeconfig ./testsys.kubeconfig logs -f x86-64-vmware-k9bba8e96-b5ea-424f-8a90-a44e0ec31e4f-creatipx52m -n testsys
[2023-12-12T23:37:41Z INFO  resource_agent::agent] Initializing Agent
[2023-12-12T23:37:42Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Getting vSphere secret
[2023-12-12T23:37:42Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Creating working directory
[2023-12-12T23:37:42Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Checking existing cluster
[2023-12-12T23:37:42Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Creation policy is 'IfNotExists' and cluster 'x86-64-vmware-k8s-128' does not exist: creating cluster
[2023-12-12T23:37:42Z INFO  bottlerocket_agents::clusters] Using EKS-A release manifest 'https://anywhere-assets.eks.amazonaws.com/releases/eks-a/manifest.yaml'
[2023-12-12T23:37:42Z INFO  bottlerocket_agents::clusters] Fetching EKS-A binary archive from 'https://anywhere-assets.eks.amazonaws.com/releases/eks-a/54/artifacts/eks-a/v0.18.3/linux/amd64/eksctl-anywhere-v0.18.3-linux-amd64.tar.gz'
[2023-12-12T23:37:42Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Creating cluster
[2023-12-12T23:37:42Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Downloading OVA 'bottlerocket-vmware-k8s-1.28-x86_64-v1.16.1.ova'
[2023-12-12T23:37:45Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Importing OVA and creating a VM template out of it
[2023-12-12T23:37:58Z INFO  vsphere_k8s_cluster_resource_agent::vsphere_k8s_cluster_provider] Tagging VM template
2023-12-12T23:38:54.138Z	V4	Reading bundles manifest	{"url": "https://anywhere-assets.eks.amazonaws.com/releases/bundles/54/manifest.yaml"}
2023-12-12T23:38:54.172Z	V4	Using CAPI provider versions	{"Core Cluster API": "v1.5.2+b14378d", "Kubeadm Bootstrap": "v1.5.2+b8987c8", "Kubeadm Control Plane": "v1.5.2+5762149", "External etcd Bootstrap": "v1.0.10+c9a5a8a", "External etcd Controller": "v1.0.16+0ed68e6", "Cluster API Provider VSphere": "v1.7.4+6ecf386"}
2023-12-12T23:38:54.173Z	V4	Relative network path specified, using path /SDDC-Datacenter/network/sddc-cgw-network-2
....

VM resource agent:

$ kubectl --kubeconfig ./testsys.kubeconfig logs -f x86-64-vmware-k14076f07-2099-4f11-b093-67085fa6f4d1-creatitkwwg -n testsys
[2023-12-12T23:44:20Z INFO  resource_agent::agent] Initializing Agent
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Initializing agent
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Checking SSM service role
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Getting vSphere secret
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Setting GOVC env
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Writing kubeconfig
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Creating K8s client
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Downloading OVA
[2023-12-12T23:44:20Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Downloading OVA 'bottlerocket-vmware-k8s-1.28-x86_64-v1.16.1.ova'
[2023-12-12T23:44:23Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Retrieving K8s cluster info
[2023-12-12T23:44:23Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Creating bootstrap token
[2023-12-12T23:44:23Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Create a bootstrap token for node registrations
[2023-12-12T23:44:23Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Updating import spec
govc: vm 'x86-64-vmware-k8s-128-node-vmtemplate' not found
[2023-12-12T23:44:23Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Importing OVA
[2023-12-12T23:44:23Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Importing OVA and creating a VM template out of it
[2023-12-12T23:44:36Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Creating template from OVA
[2023-12-12T23:44:36Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Launching worker nodes
[2023-12-12T23:44:36Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Launching 2 Bottlerocket worker nodes
[2023-12-12T23:44:36Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Cloning VM for worker node 'x86-64-vmware-k8s-128-node-d0d31b4f'
[2023-12-12T23:44:40Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Powering on 'x86-64-vmware-k8s-128-node-d0d31b4f'...
[2023-12-12T23:45:21Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Cloning VM for worker node 'x86-64-vmware-k8s-128-node-95703674'
[2023-12-12T23:45:24Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] Powering on 'x86-64-vmware-k8s-128-node-95703674'...
[2023-12-12T23:46:07Z INFO  vsphere_vm_resource_agent::vsphere_vm_provider] VM(s) created
[2023-12-12T23:46:07Z INFO  resource_agent::agent] Resource action succeeded.
[2023-12-12T23:46:07Z INFO  resource_agent::agent] 'keep_running' is true.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[ecpullen]

LGTM