#helmfile - October 2019

Hi everyone! We're using Azure Container Registries as our helm repo, as per https://docs.microsoft.com/en-us/azure/container-registry/container-registry-helm-repos

The integration with helm works fine by adding credentials to ~/.helm/repository/repositories.yaml but it seems helmfile ignores this file so we have to add credentials to helmfile.yaml thus:

# Advanced configuration: You can setup basic or tls auth
- name: roboll
  url: <http://roboll.io/charts>
  certFile: optional_client_cert
  keyFile: optional_client_key
  username: optional_username
  password: optional_password

Is there any way to get helmfile to use ~/.helm/repository/repositories.yaml or any plans to do so?

Anyone know how I can configure helmfile to install the latest pre-release versions of my charts? Tried adding --devel to args and omitting version from the releases, which does what I want when I helmfile diff, but when I helmfile apply it will try to install the latest release version.

anyone know the exact syntax of the --state-values-set flag? I can't seem to get it to work

theres also not much documentation for this

we use helmfile to deploy all our envs, but we have envs per dev team (about 12) - all pretty much with the same config except for some small uniform changes I should be able to influence with that flag I would expect, but I can't seem to get it to work

Any tips & tricks to share regarding running Helmfile from Atlantis? I’m looking into cross-account auth with EKS and it gets dicey.

I'm trying to manage our environments via helmfile and I'm running into issues. I'm trying to do common helmfiles (with release definitions) in helmfiles/ and have config in config/<env>/<proj>/*.values.yaml. Theres a base.yaml which defines repos, helm defaults, and environments (prod + stage for now). My helmfile.yaml includes the base via bases: and then has a helmfiles: directive for helmfiles/*.helmfile.yaml. The individual app helmfiles also include the base. Any values I set on environments aren't getting through to the app helmfiles. I've tried overriding them in the helmfiles: section but that just ends up failing to render because it can't find the environment value there either. If I remove the bases: from helmfile.yaml, though, and just paste the contents inline, it works. there seems to be some weird interaction with bases that i'm not understanding

I am trying out the helmfile, In base helmfile.yaml have multiple releases, And I am working on only one say cert-manager. While apply the helmfile all the applications are getting deployed.

helmfiles:
  - "releases/prometheus-operator.yaml"
  - "releases/cluster-autoscaler.yaml"
  - "releases/cert-manager.yaml"

I am using the below command

helmfile --file helmfile-preprod.yaml -e preprod apply

Is there anyway to pass argument which only deploys cert-manager.yaml?
Kindly suggest

Using

--selector value, -l value

we can run a particular release

You can also run the release helmfile directly with the --file argument

@Gourav or you can use the --selector argument

@Erik Osterman (Cloud Posse) Thanks Erik

Need some suggestion, in below section there's a env KIAM_HOST_CERT_PATH variable is there , if I am not passing any value to it, It will pickup the default one.

extraHostPathMounts:
          - name: "ssl-certs"
            mountPath: "/etc/ssl/certs"
            hostPath: '{{ env "KIAM_HOST_CERT_PATH" | default "/etc/ssl/certs" }}'
            readOnly: true

I wanted to understand, where I need to define this variable KIAM_HOST_CERT_PATH and pass its value. As I do not want to change the default values but wanted to use passed value.
Is there any example to achieve the above mentioned?

that's an environment variable

you can define it before you run helmfile or on the same line

KIAM_HOST_CERT_PATH="/path/to/certs" helmfile

@zeid Thanks.. i will try.

Hi all,
I'm attempting to override release values so that devs can quickly change image.tag on their local env for testing. The current pattern is like so;

bases:
  - ../../env/helmfile-environments.yaml

releases:
  - name: myrel
    namespace: dataproduct
    chart: ../../../../charts/data-product-service-chart
    force: true
    atomic: true
    values:
      - image:
          repository: <http://myrepo.com/rel/myrel|myrepo.com/rel/myrel>
          tag: master-1.1.123
      - ../../env/{{ .Environment.Name }}.yaml

and we want to run helmfile like this;

helmfile -e minikube -f helmfile-myrel.yaml --state-values-set image.tag=dev apply

However, the image.tag value remains as master-1.1.123

I should also say that image.tag is only defined in the Chart's values file and not in the referenced env/minikube.yaml file

Hi.. I am working on kiam helmfile where I wanted to move rbac section from below file to file named "values/kiam.yaml.gotmpl". So i have included the file as show below under values: section. But I am getting the below message. Anyone got some tips for me?

helmfile --file helmfile-dev-dev.yaml -e dev-dev -l chart=kiam diff

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read kiam.yaml.part.1: reading document at index 1: yaml: line 148: did not find expected '-' indicator
in ./helmfile-dt-ue2.yaml: in .helmfiles[0]: in releases/kiam.yaml: failed to read kiam.yaml: reading document at index 1: yaml: line 148: did not find expected '-' indicator

- name: "kiam"
  namespace: "kube-system"
  labels:
    chart: "kiam"
    repo: "stable"
    component: "iam"
    namespace: "kube-system"
    vendor: "uswitch"
    default: "true"
  chart: "stable/kiam"
  version: "2.5.2"
  wait: true
  recreatePods: false
  installed: {{ env "KIAM_INSTALLED" | default "true" }}
  hooks:
    # This hoook adds the annotation that allows pods in the kube-system namespace to assume any annotated role
    - events: ["presync"]
      command: "/bin/sh"
      args: ["-c", "kubectl annotate --overwrite namespace kube-system '<http://iam.amazonaws.com/permitted=.*|iam.amazonaws.com/permitted=.*>'"]
    # This hook adds the annotation that instructs stakater/reloader to watch the DaemonSet's secrets and configmaps
    # and reload the DeamonSet when they change.
    - events: ["postsync"]
      command: "/bin/sh"
      args: ["-c", "kubectl annotate --overwrite --namespace={{`{{ .Release.Namespace }}`}} DaemonSet --selector=app=kiam <http://reloader.stakater.com/auto=true%22]|reloader.stakater.com/auto=true"]>
  values:
    - fullnameOverride: kiam
    - values/kiam.yaml.gotmpl
      #rbac:
        ### Optional: RBAC_ENABLED;
        #create: {{ env "RBAC_ENABLED" | default "false" }}

Hi... I am preparing the helmfile for kiam. In kiam specifications there are seviceAccount section, in that when I am trying to override the serviceAccountName for agent and server it is not happening. Below is the snippet of manifest where I am trying to override

serviceAccount:
  agent:
    create: true
    name: dev-ops-kiam-agent
  server:
    create: true
    name: dev-ops-kiam-server

While doing the helmfile diff I am getting the serviceAcccount manifest files for agent and server. But in that name is not coming as dev-ops-kiam-agent and dev-ops-kiam-server instead coming as kiam-agent and kiam-server.

+ apiVersion: v1
+ kind: ServiceAccount
+ metadata:
+   labels:
+     app: kiam
+     chart: kiam-2.5.2
+     component: "agent"
+     heritage: Tiller
+     release: kiam
+   name: kiam-agent

Does someone have some pointers for me to resolve this issue with serviceAccount ?

I have a rather extensive helmfile setup, a few dozen helmfiles (all loaded from a central-one) with multiple releases per helmfile. What would be the best way to set a specific set of values for the charts in every single release accross a deploy? more specifically, we allow the hostAliases in deployments to be set in all our charts - but want this to be managed centrally... I now have to include a central file with these hostaliases in every single release, but I would like to be able to add that from a single central place (preferably from our base helmfiles (which are already included in all our helmfiles)

I tried with a template in our base role, but values arrays aren't merged, so then I can't set release-specific values anymore

Hi, I have what seems like a silly question, didn't want to open an issue on GitHub. I'm using git for a remote helmfile. But, there doesn't seem to be any way to have the git repo get updated with git pull other then going into the .helmfile directory and doing it myself?

Anyone using helmfile for multiple clusters in different regions? Whats your approach if every cluster has a location specific variable?

Just got an issue, that has never happened before. After the latest helmfile apply some of the releases became FAILED, and corresponding workloads got disappeared. In logs I can see something like

exec: helm tiller run gitlab-managed-apps -- helm upgrade --install --reset-values frontend-e2e-alpha chartmuseum/frontend --version 0.9.0 --timeout 300 --force --namespace e2e-alpha --values /tmp/values674824445 --kube-context=gke_XXXX_europe-west1-b_XXXX: Creating tiller namespace (if missing): gitlab-managed-apps
UPGRADE FAILED
ROLLING BACK
Error: Failed to recreate resource: the server was unable to return a response in the time allotted, but may still be processing the request (post deployments.apps)

After that the deployment got disappeared.

frontend-e2e-alpha      	16      	Thu Oct 17 05:23:17 2019	FAILED  	frontend-0.9.0           	1.0                         	e2e-alpha 

That’s weird. And some deployments now have pods from different revisions. Things got messed up.

I purged a bunch of releases and ran helmfile apply again. Almost all succeeded, except one.

The same didn’t help for the other bunch of releases at all.

hmm have another issue... I use

{{`{{.Release.Name}}`}}

in a template section in the values:. If it's in a filename, it renders properly, if it's in a - varname: {{...}} it doesn't render this, and I end up with a literal {{ .Release.Name }} in my templated helm chart output...

Are containers built somewhere having helm3 binary, latest work-in-progress helm-diff (including needed fixes)?

Actually, I’ve got kinda related question. Do we have any helm2 -> helm3 transition best practices for those using helmfile? Or they are pretty much the same as for Helm? Is helmfile ready to be used with Helm 3 right now? Is there any missing things? Even though I’ve been using helmfile for quite some time, I’d never tried it with Helm 3.

Hello! Just started looking at helmfile and I think I like it. 🙂 Our current setup is that we have a Jenkins pipeline for each micro service that creates a docker image and a helm chart which gets pushed/published to our registries. The pipelines also deploys to our environments based on which branch it is, but I want to move all that to a helmfile in a separate git repo, which can be updated using PRs for values changes, but I want the built helm chart deployed automatically and thus want the Jenkins pipeline to do git commits to the repo where the helmfile resides… Anyone got any good way of doing this?

Are there any up to date helmfile examples?

We use these everyday

Taking https://github.com/costimuraru/helmfile-examples/tree/master/templatization as an example, how can i move cluster-autoscaler.yaml into a sub-directory called releases? I think i keep running into path issues.

does anyone know why i’m getting this warning on a specific helmfile?

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read creds.yaml.part.1: reading document at index 1: yaml: unknown anchor 'default' referenced

i have the same structure for the rest of the helmfiles and they do not return this warning

@Erik Osterman (Cloud Posse) Is there helmfile for Open Policy Agent? I have checked in helmfiles/releases and there is none for OPA.

Even if helmfile is not there... Are we allowed to create our own helmfile for which stable helm charts are there?

Got a question. Anyone have thoughts on a workflow for modifying existing helmcharts without forking them? We have so many forks to do silly stuff like add tolerations or ssl root certs.

@Paul Becotte what is the reason for the forks? do u change the templates and functionality that the original chart does not provide? or just the values?

Changing templates. Like the public chart doesn't have 'tolerations' as a field on a deployment, and we needed to add it. (And many similar examples).

one option is just to open PR and suggest a change, toleration is a common use case imo.
the second option i can think of, is replicated ship, never used it myself but seems to fit here

Yeah, PRs of course, but waiting weeks for a public project to accept and release isn't usually in the cards

Hi

When we do helmfile apply it is printing diff output which conatins sensitive info how can we disable that.

Hi again... Need some inputs on a issue facing with Kiam helmfile where I am trying to give the annotations at object level. But somehow annotations are not coming up.. below are the snipped what I am getting and what i need.

While running the helmfile.. I am not getting the annotations at object level

+ # Source: kiam/templates/agent-daemonset.yaml
+ apiVersion: apps/v1beta2
+ kind: DaemonSet
+ metadata:
+   labels:
+     app: kiam
+     chart: kiam-2.5.2
+     component: "agent"
+     heritage: Tiller
+     release: kiam
+   name: kiam-agent
+ spec:
+   selector:
+     matchLabels:
+       app: kiam
+       component: "agent"
+       release: kiam
+   template:
+     metadata:
+       annotations:
+         <http://secret.reloader.stakater.com/reload|secret.reloader.stakater.com/reload>: kiam-agent-certificate-secret,kiam-ca-cert

Expected output should something like

+ # Source: kiam/templates/agent-daemonset.yaml
+ apiVersion: apps/v1beta2
+ kind: DaemonSet
+ metadata:
+  annotations:
+    <http://secret.reloader.stakater.com/reload|secret.reloader.stakater.com/reload>: kiam-agent-certificate-secret,kiam-ca-cert
+   labels:
+     app: kiam
+     chart: kiam-2.5.2
+     component: "agent"
+     heritage: Tiller
+     release: kiam
+   name: kiam-agent
+ spec:
+   selector:
+     matchLabels:
+       app: kiam
+       component: "agent"
+       release: kiam
+   template:
+     metadata:
+       
+         

Opened up a PR for cloudposse's nginx-ingress helmfile to add PROXY support, tested and works great in my staging cluster: https://github.com/cloudposse/helmfiles/pull/199

Is there a helmfile icon or logo somewhere? Preferably SVG.

Is there a way to treat an entire helmfile as an atomic release? I.e. rollback ALL releases in the helmfile if ANY fail?