What options and defaults exists for APIML outbound https requests

[1000TurquoisePogs]

I'm working with a user who aims to disable TLS 1.2 everywhere, but is encountering issue with APIML being then unable to connect to z/OSMF.

I see here

api-layer/gateway-package/src/main/resources/bin/start.sh

Line 269 in b8976cd

-Dapiml.httpclient.ssl.enabled-protocols=${ZWE_configs_apiml_httpclient_ssl_enabled_protocols:-"TLSv1.2"} \

And/or here

api-layer/gateway-package/src/main/resources/bin/start.sh

Line 279 in b8976cd

-Dserver.ssl.protocol=${ZWE_configs_server_ssl_protocol:-"TLSv1.2"} \

References to TLS v1.2 defaults which if I switch to "TLSv1.3", I do get different behavior.
But these aren't documented in the schema, so I don't really know what they are.

1. are they a list? can you say "TLSv1.3,TLSv1.2" ?
2. is one for requests and the other for the server listening?
3. the servers seem to default to serving 1.3 these days, so is one ignored? or are both for requests?

Hopefully the answers to these questions assist in making #3569 have great behaviors.

[balhar-jakub]

@pj892031 Can you please provide further information on the topic?

[pj892031]

If there is no strong preference for the version of TLS, it is possible to provide server.ssl.protocol = TLS.

The list of ciphers in the APIML is currently basically this one:
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384
note: Just micronaut-sample-app has any reason missing these: TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384

It is set on a general level (hardcoded in Java), see

api-layer/common-service-core/src/main/java/org/zowe/apiml/security/HttpsConfig.java

Line 36 in bd4e738

String[] cipherSuite = "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384".split(",");

and

api-layer/apiml-common/src/main/java/org/zowe/apiml/product/web/HttpConfig.java

Line 58 in bd4e738

@Value("${server.ssl.ciphers:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384}")

The list of ciphers is set again in services internal configuration (mock-service, api-catalog-service, caching-service, discoverable-client, discovery-service, onboarding-enabler-spring-micronaut-sample-app, onboarding-sample-app, gateway). Each service uses the property server.ssl.ciphers, except Micronaut one (it uses apiml.service.ssl.ciphers). But configuring the APIML services doesn't matter, because those services are just for testing purposes.

To change ciphers used by APIML is necessary to set the system environment SERVER_SSL_CIPHERS. Ciphers should be separated by a comma.

To control outgoing communication it is possible to use also configuration value apiml.httpclient.ssl.enabled-protocols

Example on how to configure zowe.yaml:

zowe:
  environments:
    SERVER_SSL_PROTOCOL: TLS
    SERVER_SSL_CIPHERS: TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
    SSL_ENABLED_PROTOCOLS: TLSv1.2,TLSv1.3
    APIML_HTTPCLIENT_SSL_ENABLED_PROTOCOLS: TLSv1.2,TLSv1.3

Note: System environment SERVER_SSL_PROTOCOL could be replaced by zowe.components.[component name].server.ssl.protocol or ZWE_configs_server_ssl_protocol (it is valid only for caching service, API catalog, discovery service, and API Gateway)
Note: The list of ciphers should be defined by each customer based on their policy.
Note: Check also IBM documentation Enabling 1.3 how to set up TLS 1.3 in Java.

---

I hope this summary could help, if not, please do not hesitate to ask for any detail.

[1000TurquoisePogs]

When I turn on gateway debugging I see the following ciphers:

ZWEAGW1:main:... ZWESVUSR INFO (o.z.a.p.w.HttpConfig) Using HTTPS configuration: HttpsConfig(protocol=TLSv1.3, enabledProtocols=[TLSv1.3], trustStore=..., trustStoreType=JCERACFKS, trustStoreRequired=false, keyAlias=..., keyStore=..., cipherSuite=[TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384],

This matches your list.

But later on, it switches to a different list:

ZWEAGW1:main:... ZWESVUSR DEBUG (o.a.h.c.s.SSLConnectionSocketFactory) Enabled cipher suites:[SSL_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384, SSL_ECDHE_RSA_WITH_AES_128_GCM_SHA256, SSL_DHE_RSA_WITH_AES_256_GCM_SHA384, SSL_DHE_DSS_WITH_AES_256_GCM_SHA384, SSL_DHE_RSA_WITH_AES_128_GCM_SHA256, SSL_DHE_DSS_WITH_AES_128_GCM_SHA256, SSL_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, SSL_ECDHE_RSA_WITH_AES_256_CBC_SHA384, SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA256, SSL_DHE_RSA_WITH_AES_256_CBC_SHA256, SSL_DHE_DSS_WITH_AES_256_CBC_SHA256, SSL_DHE_RSA_WITH_AES_128_CBC_SHA256, SSL_DHE_DSS_WITH_AES_128_CBC_SHA256, SSL_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, SSL_ECDH_RSA_WITH_AES_256_GCM_SHA384, SSL_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, SSL_ECDH_RSA_WITH_AES_128_GCM_SHA256, SSL_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, SSL_ECDH_RSA_WITH_AES_256_CBC_SHA384, SSL_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, SSL_ECDH_RSA_WITH_AES_128_CBC_SHA256, SSL_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, SSL_ECDHE_RSA_WITH_AES_256_CBC_SHA, SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA, SSL_DHE_RSA_WITH_AES_256_CBC_SHA, SSL_DHE_DSS_WITH_AES_256_CBC_SHA, SSL_DHE_RSA_WITH_AES_128_CBC_SHA, SSL_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_ECDH_ECDSA_WITH_AES_256_CBC_SHA, SSL_ECDH_RSA_WITH_AES_256_CBC_SHA, SSL_ECDH_ECDSA_WITH_AES_128_CBC_SHA, SSL_ECDH_RSA_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_AES_256_GCM_SHA384, SSL_RSA_WITH_AES_128_GCM_SHA256, SSL_RSA_WITH_AES_256_CBC_SHA256, SSL_RSA_WITH_AES_128_CBC_SHA256, SSL_RSA_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_AES_128_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]

Where does SSLConnectionSocketFactory get this list from?
It's lacking TLS 1.3 ciphers. I am guessing the first cipher list is for server, but this larger list is my problem, as it seems to be that the client requests (such as to zosmf) do not contain tls 1.3 ciphers.

[1000TurquoisePogs]

Could it be that the content within here does not honor the cipher customization? https://github.com/zowe/api-layer/blob/v2.x.x/common-service-core/src/main/java/org/zowe/apiml/security/HttpsFactory.java

[1000TurquoisePogs]

I found that adding -Djdk.tls.client.cipherSuites="TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384" had an effect. I think this should be incorporated into the configuration of apiml since I had to edit start.sh to add it.

[balhar-jakub]

It feels like we miss configuration property for ciphers if I am looking correctly on the code above and that within the API ML we need to propagate it to two places:

jdk.tls.client.cipherSuites and server.ssl.ciphers

[balhar-jakub]

The cipher property already exists in the zowe.yaml. The content needs to be converted to the state that API ML can consume.

We may be missing other places where the configuration needs to happen.

[1000TurquoisePogs]

Coincidentally was just checking on this

If you set

  network:
    server:
      tls:
        ciphers:
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384

You get

ZWE_zowe_network_server_tls_ciphers=TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384

So... pretty convenient to match up with what start.sh does.
I'll make quick update of my branch on testing such configs

[1000TurquoisePogs]

I believe this captures it, but needs testing
d723932