Disable KubeVirt live migrate during upgrades (backport #60)

[mergify]

Problem:

The LiveMigrate workload update method indicates that all VMs will be live-migrated to update their launcher images during the cluster-upgrade phase as soon as KubeVirt gets upgraded to the new version. These excessive VM live migrations are considered harmful to the cluster and upgrade stability.

Solution:

Temporarily turn off the LiveMigrate workload update method during the upgrade. Revert it at the end of the upgrade.
The harvester ManagedChart also needs to be handled to prevent Fleet from reverting the config during the upgrade.

Related Issue(s):

harvester/harvester#10349

Test plan:

1. Prepare a 3-node cluster in v1.7.1
2. Create some VMs
3. Prepare the upgrade toolkit image (either build it yourself or use mine, starbops/harvester-upgrade-toolkit:feat-disable-kubevirt-live-migrate-during-upgrade-head)
4. Install and enable the harvester-upgrade-manager add-on

   apiVersion: harvesterhci.io/v1beta1
   kind: Addon
   metadata:
     name: harvester-upgrade-manager
     namespace: harvester-system
     labels:
       addon.harvesterhci.io/experimental: "true"
   spec:
     enabled: true
     repo: https://charts.harvesterhci.io
     version: 1.8.0-rc6
     chart: harvester-upgrade-manager
     valuesContent: |
       manager:
         image:
           repository: starbops/harvester-upgrade-toolkit
           tag: feat-disable-kubevirt-live-migrate-during-upgrade-head

5. Create the Version CR

   apiVersion: harvesterhci.io/v1beta1
   kind: Version
   metadata:
     name: v1.8.0-rc6
     namespace: harvester-system
   spec:
     isoURL: https://releases.rancher.com/harvester/v1.8.0/harvester-v1.8.0-rc6-amd64.iso
     isoChecksum: "daf8e33f402a80640df5f4b8de22eb496e6c29f98ab29a9ee1adaef9dd5fc355e2fef547f5c2a5a1f60d541c09e6930a220a73f013d2368eed51f9ea174f6d96"

6. Create the UpgradePlan CR

   apiVersion: management.harvesterhci.io/v1beta1
   kind: UpgradePlan
   metadata:
     annotations:
       management.harvesterhci.io/allow-deletion: "true"
       management.harvesterhci.io/upgrade-toolkit-image: starbops/harvester-upgrade-toolkit
     generateName: hvst-upgrade-
   spec:
     version: v1.8.0-rc6
     upgrade: feat-disable-kubevirt-live-migrate-during-upgrade-head

7. During the cluster-upgrade phase, the KubeVirt CR should be patched (LiveMigrate removed), and the harvester ManagedChart CR should be patched with the corresponding comparePatch
8. As soon as KubeVirt gets upgraded, observe that there are no *-workload-update-* VMIMs created and no VMs are being live migrated
9. The upgrade should complete successfully
10. After the upgrade completes, both the KubeVirt CR and harvester ManagedChart should be reverted

Additional documentation or context

Porting changes from harvester/harvester#10386

Note

The major difference between harvester/harvester#10386 is that LiveMigrate disabling is handled in the controller using a pure Go implementation instead of a shell script.

---

This is an automatic backport of pull request #60 done by Mergify.