Space Automation Simple CD Pipeline
When it will be possible to have jobs dependent on each other? Something like Gitlab stages, when a result of a previous stage matters. How could someone build a proper CD pipeline with current automation elements?
Let me elaborate a bit... I want to run 3 simple, but dependent things sequentially:
BUILD & TEST (automatic) -> PUBLISH (automatic) -> DEPLOY (manual).
I can define 3 jobs for that, but there is no relation between the jobs, they cannot be executed sequentially (from my understanding), so It will be possible to run DEPLOY before PUBLISH gets executed or even if it fails, which does not make sense of course.
Or I can define only 1 job with multiple sequential steps, I can pass artifacts to the next step, etc., but there is no way to manually invoke the last DEPLOY step, right? Steps cannot be invoked manually (only jobs can). Please, correct me if I am wrong.
So, my question is, how to do such a simple and common thing with the current Automation API? Thank you.
Please sign in to leave a comment.
Hi Jan Tobola, thank you very much for sharing your use case with us. In fact, our team is already evaluating the best way we could address such scenarios in Space Automation. Please feel free to subscribe to the following issue to be updated on the progress:
https://youtrack.jetbrains.com/issue/SPACE-13604
Hey Jan Tobola, we're going to specifically support your exact use case with a Pipelines feature. It's going to introduce stages-like with a dependencies between jobs, and also "Gates" concept that will allow to guard some parts of pipeline a pipeline and will require a manual action to proceed. You can subscribe to this issues to track the progress: https://youtrack.jetbrains.com/issue/SPACE-10801, the ETA is Q4 2022.
For now, one workaround that I can think of is to put `BUILD & TEST (automatic) -> PUBLISH (automatic)` in first job, and deploy into second. Then you can subscribe to the first job results either in your personal feed or in some channel, and trigger second job manually based on notifications when you see fit (check docs here: https://www.jetbrains.com/help/space/viewing-job-execution-results.html).
Alexander Demukh Is there any ETA update for the pipeline feature?
Aleksandr Tochinov Hey! The pipelines feature got pushed back. We're hoping to deliver it somewhere in 2023, but there's not hard ETA that I can share at the moment. The delay is partially due to ability to emulate pipelines using regular jobs, just as I've described above (you can extract "deploy" step into a separate job and run it separately, there's also ability to automate the process by triggering the "Deploy" job through the Space API). Also, we've released new parameters engine and Custom Run UI (https://www.jetbrains.com/help/space/automation-parameters.html). This is a necessary backend part for a pipelines feature as well. This new feature now fundamentally fixes the problem in an original question, you now can have a parameterised "Deploy" job where you can pass the artifact that should be deployed along with any other custom parameters.
Thanks for response. We have already implemented the workaround you are describing and used the features you described. But from my point it look's like implementing(reinventing) pipeline by my own. It doesn't feel like a native solution.
Looking forward pipelines to be implemented and released.