Codefresh Variable Substitution

JeremyCodefreshLeave a Comment

It’s important to understand the behavior of variables like ${{CF_PULL_REQUEST_NUMBER}} in Codefresh pipelines and how it interacts with shell commands.

When you use ${{CF_PULL_REQUEST_NUMBER}} in a YAML file, Codefresh handles it like this:

  • If the variable is set, the whole thing is replaced with the value of the variable
  • If the variable is not set, it is left unchanged.

When you execute a command in a freestyle step, that command is passed to bash. So when CF_PULL_REQUEST_NUMBER is set to something like 34:

{!{code class=""}!}czo1MDpcImNmX2V4cG9ydCBJTUFHRV9UQUc9cHItJHt7Q0ZfUFVMTF9SRVFVRVNUX05VTUJFUn19XCI7e1smKiZdfQ=={!{/code}!}

becomes: cf_export IMAGE_TAG=pr-34, but when the variable is not set, the command is:
cf_export IMAGE_TAG=pr-${{CF_PULL_REQUEST_NUMBER}}
which of course is bad bash variable interpolation syntax.

We (Cloud Posse) have created a tool to help with this called require_vars that we ship in geodesic. If your pipeline should not run if certain variables are not supplied, you can create an initial step like this:

<button class="sw-snippet-button" data-label-copy="Copy" data-label-copied="Copied">Copy</button>{!{code class="json"}!}czo4ODU6XCJ2YWxpZGF0ZToKICAgIHRpdGxlOiBcIlZhbGlkYXRlXCIKICAgIGRlc2NyaXB0aW9uOiBcIkVuc3VyZSBidWlsZCBwYXJhbWV7WyYqJl19dGVycyBhcmUgcHJlc2VudFwiCiAgICBzdGFnZTogUHJlcGFyZQogICAgaW1hZ2U6IGNsb3VkcG9zc2UvZ2VvZGVzaWM6MC4xMTYuMAp7WyYqJl19ICAgIGVudHJ5X3BvaW50OiAvZXRjL2NvZGVmcmVzaC9yZXF1aXJlX3ZhcnMKICAgIGNtZDoKICAgICAgLSB8LQogICAgICAgICR7e3tbJiomXX1HSVRIVUJfUkVQT19TVEFUVVNfVE9LRU59fSBQZXJzb25hbCBBY2Nlc3MgVG9rZW4gdXNlZCB0byBnaXZlIHNjcmlwdHMKICAgICAge1smKiZdfSAgcGVybWlzc2lvbiB0byB1cGRhdGUgdGhlIFwic3RhdHVzIGNoZWNrXCIgc3RhdHVzIG9uIEdpdEh1YiBwdWxsIHJlcXVlc3RzCiAgIHtbJiomXX0gICAtICR7e1NMQUNLX1dFQkhPT0tfVVJMfX0gU2VjcmV0IFVSTCB1c2VkIGJ5IHNjcmlwdHMgdG8gc2VuZCB1cGRhdGVzIHRvIGEge1smKiZdfVNsYWNrIGNoYW5uZWwKICAgICAgLSB8LQogICAgICAgICR7e0FXU19ET0NLRVJfUkVQT19IT1NUfX0gVGhlIGhvc3QgaGFtZSBwb3J7WyYqJl19dGlvbiBvZiB0aGUgRUNSIERvY2tlciByZXBvIHRvIHVzZS4KICAgICAgICBUeXBpY2FsbHkgc29tZXRoaW5nIGxpa2UgMTIzNDU2N3tbJiomXX04OTAxMi5ka3IuZWNyLnVzLWVhc3QtMS5hbWF6b25hd3MuY29tCiAgICAgIC0gfC0KICAgICAgICAke3tDRl9QVUxMX1JFUVVFU1Rfe1smKiZdfU5VTUJFUn19IFRoZSBQUiBudW1iZXIgZnJvbSBHaXRIdWIuCiAgICAgICAgVGhlIFBSIG51bWJlciBpcyBvbmx5IHNldCBpZiB0aGl7WyYqJl19cyBidWlsZCB3YXMgdHJpZ2dlcmVkIGluIHJlbGF0aW9uIHRvIGEgUFIuCiAgICAgICAgUmVxdWlyaW5nIHRoaXMgdG8gYmUgcHJlc3tbJiomXX1lbnQgbWVhbnMgcmVxdWlyaW5nIHRoaXMgcGlwZWxpbmUgdG8gb25seSB3b3JrIHdpdGggUFJzLlwiO3tbJiomXX0={!{/code}!}

The PR number is only set if this build was triggered in relation to a PR.
Requiring this to be present means requiring this pipeline to only work with PRs.

What this step will do is verify the referenced variables are set and output the text as an error if they are not, causing the build to fail at that step. We recommend using it whenever a pipeline would not function properly with missing variables.

The other thing is that if a variable is optional, you test for it being set or not by checking to see if the value of the variable contains the name of the variable. So, if a step should only run if there is a PR number assigned, you can add a test like this:

<button class="sw-snippet-button" data-label-copy="Copy" data-label-copied="Copied">Copy</button>{!{code class=""}!}czoxMTg6XCJ3aGVuOiAKICBjb25kaXRpb246IAogICAgYWxsOiAKICAgICAgaXNQUjogXCdpbmNsdWRlcyhcIiR7e0NGX1BVTExfUkVRVXtbJiomXX1FU1RfTlVNQkVSfX1cIiwgXCJDRl9QVUxMX1JFUVVFU1RfTlVNQkVSXCIpID09IHRydWVcJ1wiO3tbJiomXX0={!{/code}!}

(It may seem, and really should be, redundant to have == true but it seems to be required.)

It is kind of weird and confusing but makes sense once you get the hang of it. Please feel free to reach out to me, if you have further questions.