Designing CI/CD like Github or Gitlab
Hello. I tried to design more advanced CI/CD using Automation feature. I met some problems which I'm not sure how to resolve. Perhaps, I don't understand the idea behind Automation, but I find it not intuitive for designing CI/CD.
I want my CI/CD to have for example 3 jobs (for simplicity):
1. Build Docker images:
- start only when necessary:
* on "main" build "my-image:lastest" only if "Dockerfile" changed or image does not exist
* on any "branch" build "my-image:my-branch" only if "Dockerfile" changed, otherwise don't build anything
- pass image tag name to other jobs to make them pull the right one
2. Run linters (using "my-image:<latest or branch name>)
3. Run tests (using "my-image:<latest or branch name>)
The problem is "Jobs" in Space are something different that we know from other systems like Github or Gitlab. They are independent from each other as you cannot specify the order and dependencies/constraints between them.
So, I think the idea is to run whole CI/CD within one Job and do everything inside it. The problem is you cannot specify when to run "host"/"container" to make 1. possible (or at least I don't know how to do that).
Can you advice anything?
Please sign in to leave a comment.
Hi Chris! Sorry for a long reply. It's a really good question. Ultimately, this case is going to be covered by our own concept of Pipelines (this concept would be much closer to GitHub and Gitlab workflows). We are planning to release Pipelines later this year, here's an issue https://youtrack.jetbrains.com/issue/SPACE-10801/Automation-Introduce-Pipelines.
There's one missing feature that we have to provide for a workaround, and we're going to fix it quicker - https://youtrack.jetbrains.com/issue/SPACE-13903/Automation-Provide-list-of-commits-received-with-a-push-as-a-parameter-to-Automation-job-run-triggered-by-git-push. With a list of commits that did change on push it would be pretty easy to track changed files using `kotlinScript` and Space API and use parameters to decide if you need building new image or not. You can subscribe to the issue and receive notification when it's ready.
Conditional steps are also not supported at the moment - https://youtrack.jetbrains.com/issue/SPACE-13604/Automation-Step-filters. I can't give you an ETA for that feature because it's unclear how much of use cases are going to be covered by Pipelines.