[OnPremKubernetes] How to use Computeservice? Follow
Hey, I followed the docs on enabling computeservice and have been able to get it to manage the automation namespace but now I'm stuck at getting them to actually accept jobs from Space.
It currently seems like Computeservice and Space don't really communicate with each other.
For one, do I set the default worker pool to Space Automation Cloud or to Self-Hosted Workers?
How to I correctly refer to the computeservice in the requirements section of the .kts-file?
Please sign in to leave a comment.
Alright, I did some testing and checked the logs of multiple workers, and found that the worker pods can't connect to http://jb-space-computeservice.kube-space:9098.
I spun up a busybox and tried to connect to that URL from there but wasn't able to reach it, not even from the same namespace.
I'm checking my cluster-config and cni next
Current error message the workers write:
Tried to access the URL from the computeservice-pod using curl:
I get the same when using the following:
When I was playing around with automation during Beta, the packages-pod was the one that provided those automation-worker.zip-files, did that change?
This is the error the computeservice-pod writes when it starts a worker:
It doesn't crash with this error though...
Till Rother could you clarify which image version is specified in your values.yaml for the compute service pod?
It's this one:
Till Rother thanks! Am I right that it's taken from the values.yaml of a default Helm chart? Could you also check what the value is set for computeservice.worker.image.tag there? Finally, please share the computeservice part of your custom values.yaml that is supposed to be created as per the following guide:
That is the customized version, the compute-service is the default one:
Till Rother, could you please explain which image is used here?
Is it a default 2023.1.0.684 one (from the latest chart), but stored in your private registry? Does your use case require customizing images for some reason? Thanks!
Pavel Boger it's the default one, just with one additional ca added to the keystore in $JAVA_HOME/lib/security/cacerts.
Pavel Boger <- this one seems to be the real one
There are two of you, don't know which one is the real one.
I managed to find a better solution to the custom-ca-problem of ours.
Instead of modifying the image, I added another subPath to the worker-config in values.yaml:
After that it works and accepts jobs from space, handles them and reports back. Now I just need to write a job that actually reports something back, as the "Hello-World"-Jobs still report as "Unable to process script" in Space. I think that just is because they don't actually write any logs.
Hi Till Rother, haha, both of them are mine, but the second one is personal :)
Thanks for the info provided. I'll check this case with the team and get back to you with the updates shortly. Stay tuned!
I'm still fiddling around with these subPaths, but am now suspecting that they refer to volumemounts from space-automation-worker:docker-inside-automation-worker
Seems like the certs are not properly passed through, the workers and then the container inside them are unable to connect to https://packages.*** because it uses a unknown ca.
Till Rother, it turned out there's a validation of the image version inside Space, therefore, the `latest` tag doesn't work in your case. Could you please republish this image with the `2023.1.0.684` tag and let me know about the results? We're going to get rid of this limitation in further versions.
Pavel Boger, thanks for confirming my suspicions :D I already did so and am now facing a new issue, coming in the form of the pre-packed *.zip files in "/home/space/circlet-server-onprem/lib/plugins/pipelines/worker/" in the space-pods. I'm assuming the workers started by computeservice are downloading those from there? They contain another java install, including another instance of cacerts-keystore. I already had them modified in the beta install and am going to try that again for those ones.
EDIT: Didn't seem to do the trick, still getting java error in the script log:
Till Rother, could you please check if custom cacerts are included in the following image as well?
Hi Pavel Boger
I didn't modify any of the other images, but will try that next, thanks! Will report back :)
I was finally able to test your suggestion and the computeservice + workers now work as expected! Thank you for your continuous support!