Correct authN and authZ configuration in Helm chart

[JakeSCahill]

What happened?

Currently, when auth.sasl.enabled: true is set in the Helm chart, it results in both enable_sasl=true and kafka_enable_authorization=true being set, which is not aligned with the recommended configuration. According to the best practices for configuring authN and authZ, the Helm chart should adhere to the following configurations:

• Option 1: Enable SASL without setting Kafka authorization
  enable_sasl=true
kafka_enable_authorization=null
authentication_method on the listeners should NOT be set
• Option 2: Use authentication_method with Kafka authorization enabled
  enable_sasl=false
kafka_enable_authorization=true
authentication_method on the listeners MUST be set

Combinations where enable_sasl=true and kafka_enable_authorization is explicitly set to true or false, or where authentication_method is set without enabling Kafka authorization, are not recommended and can lead to issues.

Additionally, when enabling SASL on an existing instance, updating the Helm values and performing an upgrade is insufficient. The changes need to be activated imperatively using rpk.

What did you expect to happen?

AuthN and auth Z should be configured correctly. See https://redpandacommunity.slack.com/archives/C01AJDUT88N/p1721169800647159?thread_ts=1721075726.706739&cid=C01AJDUT88N

Also see the docs: https://docs.redpanda.com/current/manage/security/authentication/#enable-sasl-authentication

How can we reproduce it (as minimally and precisely as possible)?. Please include values file.

resources:
  cpu: 
    cores: 1
  memory:
    container:
      min: 2Gi
      max: 3Gi
auth:
  sasl:
    enabled: true
    secretRef: "redpanda-superusers"
    users: []
external:
  type: LoadBalancer
  domain: xxx.yyy
statefulset:
  replicas: 3
  initContainers:
    setDataDirOwnership:
      enabled: true
  startupProbe:
    initialDelaySeconds: 100
storage:
  persistentVolume:
    size: 30Gi
connectors:
  enabled: false
tls:
  enabled: true
  certs:
    external:
      issuerRef:
        name: letsencrypt
        kind: ClusterIssuer
      caEnabled: false
config:
  cluster:
    default_topic_replications: 3
    minimum_topic_replications: 3
    auto_create_topics_enabled: true
  tunable:
    # these values are defined in this manifest merely for documenting purposes
    # Redpanda in Kubernetes does not apply these changes
    # see official Redpanda Helm chart documentation
    log_retention_ms: -1
listeners:
  admin:
    external:
      default:
        enabled: false
    tls:
      enabled: false
  rpc:
    tls:
      enabled: false
  http:
    authenticationMethod: none
    external:
      default:
        enabled: false
    tls:
      enabled: false
  schemaRegistry:
    authenticationMethod: none
    external:
      default:
        enabled: false
    tls:
      enabled: false
  kafka:
    tls:
      enabled: false
    authenticationMethod: none
    external:
      default:
        tls:
          enabled: true
        authenticationMethod: sasl
console:
  config:
    kafka:
      sasl:
        enabled: false

Anything else we need to know?

No response

Which are the affected charts?

Redpanda

Chart Version(s)

Latest

Cloud provider

JIRA Link: K8S-293