Configuring on-prem to use AWS resources
A few questions:
1. Can someone please go into some more detail about configuring the ingress? I have alb setup as the ingress controller.
> "Configure the TCP proxy for the VCS Ingress, namely, configure the Ingress Controller map." - not sure what to do here
2. I've set the value.yaml to include info about available redis, postgres and elastic. For some reason Im still seeing messages like this though:
Normal Pulling 15m kubelet Pulling image "docker.io/bitnami/postgresql:11.12.0"
Normal Pulled 15m kubelet Successfully pulled image "docker.io/bitnami/postgresql:11.12.0" in 8.63441786s
Normal Created 15m kubelet Created container check-postgresql
Normal Started 15m kubelet Started container check-postgresql
Normal Pulling 15m kubelet Pulling image "docker.io/bitnami/redis:6.2.7"
Normal Pulled 15m kubelet Successfully pulled image "docker.io/bitnami/redis:6.2.7" in 4.09684414s
Please sign in to leave a comment.
Looks like the images above were just to confirm that various services are available.
Im getting this exception from the jb-space-packages pods:
Caused by: org.elasticsearch.client.ResponseException: method [GET], host [http://myhost-w5l5ylotsoflettersherefl4.us-west-2.es.amazonaws.com:443], URI [/], status line [HTTP/1.1 400 Bad Request]
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
</body>
</html>
AWS opensearch provides an https url. Looks like you're expecting http. I tried setting packages.elastic.search.prefix to "https" (as a guess) to no avail.
Full text of exception:
2023-01-25 19:11:43.985 [DefaultDispatcher-worker-1] ERROR circlet.platform.a.m.a.ElasticIndex [] - Failed to create index circlet.packages.v1: Failed execution
org.elasticsearch.common.util.concurrent.UncategorizedExecutionException: Failed execution
at org.elasticsearch.common.util.concurrent.FutureUtils.rethrowExecutionException(FutureUtils.java:80)
at org.elasticsearch.common.util.concurrent.FutureUtils.get(FutureUtils.java:72)
at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:112)
at org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:100)
at org.elasticsearch.common.util.concurrent.BaseFuture.setException(BaseFuture.java:149)
at org.elasticsearch.common.util.concurrent.ListenableFuture.onFailure(ListenableFuture.java:147)
at org.elasticsearch.client.RestHighLevelClient$5.onFailure(RestHighLevelClient.java:2756)
at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:686)
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:409)
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:393)
at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.concurrent.ExecutionException: org.elasticsearch.client.ResponseException: method [GET], host [http://myhost-w5l5ylotsoflettersherefl4.us-west-2.es.amazonaws.com:443], URI [/], status line [HTTP/1.1 400 Bad Request]
Ethrbunny, thanks for the questions!
1. Note that ALB doesn't support Ingress TCP Proxy Mode. Please consider using NLB instead.
2. It seems the corresponding field is missing in the values.yaml right now, we'll add it in the future, meanwhile, please add the following values directly to JavaOpts as a workaround:
Please let us know about the results.
2023-01-26 16:08:35.306 [DefaultDispatcher-worker-1] ERROR circlet.platform.a.m.a.ElasticIndex [] - Failed to create index circlet.packages.v1: Elasticsearch version 6 or more is required
org.elasticsearch.ElasticsearchException: Elasticsearch version 6 or more is required
AWS OpenSearch is supposed to be version compatible with elastic v7
Sounds like this issue can fork between the version issue and an issue with invalid certs; here's the error I'm getting after implementing the elastic scheme:
Caused by: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Updated setup to include elasticsearch v7.10.. got this exception:
2023-01-26 17:12:04.811 [DefaultDispatcher-worker-1] ERROR circlet.platform.a.m.a.ElasticIndex [] - Failed to create index circlet.packages.v1: Invalid or missing build flavor [oss]
org.elasticsearch.ElasticsearchException: Invalid or missing build flavor [oss]
Appears to not want to work with OSS version of elastic. https://github.com/elastic/elasticsearch/issues/76091
Hopefully there's a workaround for this?
Just saw this in the setup guide:
Elasticsearch 7.11.1 – 8.2.3
I might be SOL
Went ahead and spun up elastic in ec2. Managed to get all the pods running this time.
Now to figure out the whole tcp proxy piece. Not seeing anything at <portal|packages|git>.space.local
Using nginx ingress controller - dealing with a 308 redirect loop now.. getting this working is quite a trial.
Ethrbunny could you please share your configuration? I found quite a similar issue described here.
There is no ExternalName service afaict. All those definitions are coming from the jb-space helm chart.
To get around this I had to change both service ports to https.
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: https
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
This brings seems to enable the login page. If you enter the default admin creds and try to sign in it doesn't seem to do anything. Looking at the JS console there's an error:
"Mixed Content: The page at 'https://space-portal.site.com/sign-in' was loaded over HTTPS, but requested an insecure form action 'http://space-portal.site.com/oauth/auth?response_type=token&client_id=circlet-web-ui&redirect_uri=https%3A%2F%2Fspace-portal.site.com%2Fiframe-auth.html&scope=%2A%2A&state=circlet-web-ui_state_7'. This request has been blocked; the content must be served over HTTPS."
You can get past this by removing the trailing "sign-in" from the url...or just edit the request to make it https. Still trying to figure out why that isn't being redirected.
Another thing: seems the certs that aws/acm hands out aren't going to work here. They give errors like
controller.go:1347] Unexpected error validating SSL certificate "jb-space/tls-space-packages.site.com" for server "space-packages.site.com": x509: certificate relies on legacy Common Name field, use SANs instead
This from my attempt to create a *.site.com cert and add it to the nginx ingress controller. Had to make a self-signed.