VMware : Scheduling Code Stream Pipeline Executions using Kubernetes CronJobs

In this blog post I'm going to show how I used a Kubernetes CronJob and the Code Stream CLI to schedule the execution of a Code Stream Pipeline. We use Code Stream to rebuild our demo environments every morning, the pipelines are currently executed by a scheduled script. Being able to use a CronJob means we can manage the configuration as code and automate the execution with job history.

Getting an API Token

It's always best practice to use the minimum required priviliges, so I'll generate an API token scoped to "Code Stream Executor" role (if you want to list pipelines and executions using the CLI you'll also need the Viewer, or User role):

Using the Code Stream CLI

The first step is to set up the CLI, I'm going to use the Docker image (install docs for Code Stream CLI). To do this I'll download the latest version of the CLI container image, and set up some environment variables for authentication. (For on-premesis authentication you'll need to use different variables, see the configuration documentation for Code Stream CLI)

Now we can use the CLI to look for the Pipeline ID, and grab the input form to execute it:

The input form will be different for each pipeline, depending on the inputs. The JSON format outputted by the CLI needs to be flattened to a single string and the quotes escaped so that it can be used in the command line later - e.g. the output above becomes

Create Kubernetes components

Create a namespace in which to run the CronJobs, and a secret to contain the environment variables used by the container:

Next I create the CronJob itself, which has a (when to run the task) and a spec (which defines the task to run).

• conforms to standard cron formatting, this example will execute the pipeline every 5 minutes
• is set to which means the jobs won't run concurrently
• definition will execute the CLI container image
  • is the command to execute on the container using the container ID and inputs JSON
  • creates environment variables from the secret created earlier

The CronJob YAML is applied using . We can view the and to check on the execution, and view the logs of the (remember you might need to wait until the 5-minute mark, or whatever your schedule is!)

I can see from the logs output that the execution was started successfully - and if I look in the Executions page of the Code Stream console, I can see the executions running on schedule: