【Proposal】Install Multiple Aeraki in a Single Cluster

[tanjunchen]

Is your feature request related to a problem? Please describe.

We can install multiple istio controlplanes in a kubernetes cluster, which is not currently supported by Aeraki.
We can provide a new feature that allows multiple Aeraki to be installed in single kubernetes cluster.
Multiple users do not affect each other and are softly isolated.

Describe the solution you'd like

In order to allow multiple Aeraki to be installed in the same kubernetes cluster.
All of the implementations can not affect Aeraki single clusters.
To support this feature, we may need to consider the following:
It may be incomplete, welcome to add.

• We need to support Aeraki can be installed in a specified namespace other than istio-system. make aeraki address configurable #327 aeraki support specified namespace #326
• Aeraki is only responsible for its own EnvoyFilter. EnvoyFilter of other Aeraki cannot be deleted/updated/listed.
• Aeraki should only create the sidecar bootstrap configmap in the managed namespace.
• Aeraki should only deal with ServiceEntry associated with itself.
• Aeraki starts with permissions(ClusterRole/ClusterRoleBinding etc.) isolated from each other.
• We need to think about aeraki replace the proxy images of applications managed by different istiod if multiple istio controlplanes are installed in the same kubernetes cluster.
• We may be need to think about the native Istio CRD isolation, such as dr/vs/gateway and so on.
• We also may be need to consider aeraki CRD isolation. It may be necessary for metarouter to be exported to a specified namespace.
• We need to consider validatingwebhookconfigurations of different aeraki.

We can alternatively implement this new feature once the above issues are resolved.

Provide usage documentation if the new feature is ready.

• Provide e2e case
• Provide new feature documentation

Describe alternatives you've considered

1. Modify istio source code, istio is truly multi-tenant, which is not realistic.
2. Adapt to Istio's latest soft isolation solution, see multiple-controlplanes, which is more suitable and recommended.

Additional context

• see Istio Multi-Tenancy RFC
• see multiple-controlplanes, This guide walks you through the process of installing multiple Istio control planes within a single cluster and then a way to scope workloads to specific control planes.
• see https://istio.io/latest/docs/ops/deployment/deployment-models/#tenancy-models

[zhaohuabing]

To support multi-tenancy in Aeraki's control plane, we need to create a design document. This is because it involves various parts and changes. Here are some of the things I've identified so far, but it may not be a complete list:

• We can now install Aeraki in a specified namespace other than istio-system, thanks to a recent pull request.
• It may be necessary for metarouter to be exported to a specified namespace, but this is not a hard requirement.
• We should only create the sidecar bootstrap configmap in the managed namespace.

I am curious to know if Istio has full support for multi-tenancy. When I last checked Istio's code, I found that changes to the sidecar injection webhook and CA configmap generation were required to support multi-tenancy.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.