Multi-stage cluster addons: Patching clusteraddon objects seems to cause issues

[janiskemper]

/kind bug

What steps did you take and what happened:
Sometimes when testing it happens that an upgrade flow is triggered right after the create flow. This apparently has something to do with the "ready: true" not being patched correctly. At least in the next reconcile loop it is not there anymore. Or is this cached in the next reconcile loop? The conditions are up to date.

What did you expect to happen:
This should be patched correctly and we should not trigger upgrade flows if not necessary.

[janiskemper]

We have tracked this down. And don't understand it at all. I have asked in Slack whether someone else has an idea. We patch the ClusterAddon object at the end and set the boolean "ready" on true. We also set a condition "HelmChartApplied". Then we fetch the object again and the condition is there, but the boolean is on false.

This leads to the controller thinking it is in the middle of an ongoing process and triggers a second upgrade flow, after the create flow is done. In practice, this won't harm as long as the ClusterStack is constructed well. We have actually realized this through a wrongly constructed ClusterStack where the upgrade process triggered after the create process failed.

We lack an idea of how this can happen, and it is no problem in practice. It will just do two rounds of applying the same objects. Therefore, we will leave this as is now. Hopefully, we'll find out why this happens eventually.