#helmfile - July 2019 | Slack Archive

73 messages

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/

Jun 2019

July 2019

Aug 2019

Andrew Nazarovover 6 years ago(edited)

One more question about {{'{{}}'}} syntax.
Say, I set something like this in helmfile.yaml


values:
  - hostname: {{`{{.Release.Namespace}}`}}.<http://my-domain.com|my-domain.com>

But this causes the error during helmfile lint: reading document at index 1: yaml: line 241: did not find expected key.
Is this an expected behaviour?
Btw, I’m on quite old version: v0.69.0

Micalover 6 years ago

does helmfile support creation of standalone kubernetes resources? for example if i add istio as a release and want to install a gateway, but without it being contained in a helmchart

Erik Osterman (Cloud Posse)over 6 years ago

@Mical Helmfile just calls helm. But you’re in luck...

Erik Osterman (Cloud Posse)over 6 years ago

There is a “raw” chart that does exactly what you want

Erik Osterman (Cloud Posse)over 6 years ago

Here is a example: https://github.com/cloudposse/helmfiles/blob/master/releases/external-storage.yaml

Erik Osterman (Cloud Posse)over 6 years ago

And we used it also to install an istio gateway

Erik Osterman (Cloud Posse)over 6 years ago

Here is an example of that: https://github.com/cloudposse/example-app/blob/master/deploy/releases/istio.yaml

Micalover 6 years ago

@Erik Osterman (Cloud Posse) thanks I'll check it out!

Micalover 6 years ago

What's the road map for 1.0.0? We're looking at different tools to use for decentralizing helm chart development while having a centralized way to test, package and continuously deploy systems containing a variety of charts. So far we've been looking at flux and helmfile. Flux is great but very opinionated while helmfile gives us the freedom of control since it is stateless. The fact that helmfile is still <1.0 is an issue for us for obvious reasons.

Erik Osterman (Cloud Posse)over 6 years ago(edited)

I think it's unfair to judge based on the release version being pre 1.0. Terraform is 0.12 is in use in production by enterprises/banks/etc. Obviously the uptick in traction is a wee bit less for helmfile. From my POV, better indicators to look at is how often the software is released (aka maintained), how responsive are the maintainers to issues, the overall traction of the project/community.

Erik Osterman (Cloud Posse)over 6 years ago

when you look at these characteristics, then helmfile shines.

Erik Osterman (Cloud Posse)over 6 years ago

btw, @mumoshu has now written an HelmfileOperator so you can do flux-like things with helmfile (that is a proof of concept though.

Erik Osterman (Cloud Posse)over 6 years ago

https://github.com/mumoshu/helmfile-operator

Micalover 6 years ago

@Erik Osterman (Cloud Posse) that's a valid point.

mumoshuover 6 years ago

Thanks Erik!

@Mical If I have anything to add, there would be only two things:

(1) Many orgs including my company uses helmfile in production

https://github.com/roboll/helmfile/blob/master/USERS.md

(2) Even though it is pre-1.0, Helmfile has never introduced breaking changes to existing feature for a year or so.

We do introduce breaking changes to experimental features but even those happen only after prior discussions with the known users of the feature.

Micalover 6 years ago

can i get the exit code from hooks to propagate so that helmfile sync doesn't exit with 0 on failure?

eviln1over 6 years ago

hey, just joined the slack, but been using helmfile for a 6+ months; thanks alot for making it

eviln1over 6 years ago

i've got a quick question: are environments: [] propagated to children helmfiles: [] ? looks that they aren't from my experiments, and not sure if its by design

eviln1over 6 years ago

also trying stuff out with bases: [], but i'm probably getting something wrong

Benover 6 years ago(edited)

Hi, Hoping someone knows off the top of their head why this wouldn't work


templates:
  dataproduct: &dataproduct
    namespace: dataproduct
    chart: chartmuseum/cdp-chart
    version: 1.0.1
    values:
      - app
          name: {{`"{{.Release.Name}}"`}}
      - {{ .Environment.Name }}.yaml

releases:
  - name: my-dp
    <<: *dataproduct

  - name: another-dp
    <<: *dataproduct

Benover 6 years ago(edited)

Specifically


- app:
  name:
    {{`"{{.Release.Name}}"`}}

Benover 6 years ago

It seems to be an issue with nesting values because if I change it to


values:
  - appName: {{`"{{.Release.Name}}"`}}

it doesn't fail

Benover 6 years ago

The error is

YAML parse error on cdp-chart/templates/rbac.yaml: error converting YAML to JSON: yaml: invalid map key: map[interface {}]interface {}{".Release.Name":interface {}(nil)}

mumoshuover 6 years ago

@Ben could you copy-paste your actual template once again? I see a trailing : is missing after app in https://sweetops.slack.com/archives/CE5NGCB9Q/p1563858986015700

mumoshuover 6 years ago(edited)

anyways this seems to work without emitting such error on my machine:


templates:
  dataproduct: &dataproduct
    namespace: dataproduct
    chart: chartmuseum/cdp-chart
    version: 1.0.1
    values:
      - app:
          name: {{`"{{.Release.Name}}"`}}
      - {{ .Environment.Name }}.yaml

releases:
- name: myapp
  chart: stable/mysql
  namespace: myapp
  <<: *dataproduct

Benover 6 years ago

@mumoshu Sorry, yes the original has the colon


environments:
  default:
    values:
    - default.yaml
  production:
    values:
    - production.yaml

templates:
  dataproduct: &dataproduct
    namespace: dataproduct
    chart: chartmuseum/cdp-chart
    version: 1.0.0
    values:
      - app:
          name: {{`"{{.Release.Name}}"`}}
      - {{ .Environment.Name }}.yaml

releases:
  - name: california-sos
    <<: *dataproduct

  - name: rdc
    <<: *dataproduct

Benover 6 years ago


helm version
Client: &version.Version{SemVer:"v2.14.2", GitCommit:"a8b13cc5ab6a7dbef0a58f5061bcc7c0c61598e7", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.2", GitCommit:"a8b13cc5ab6a7dbef0a58f5061bcc7c0c61598e7", GitTreeState:"clean"}

mumoshuover 6 years ago

@Ben Which helmfile version are you using?

Benover 6 years ago(edited)


> helmfile -v
helmfile version v0.80.1

mumoshuover 6 years ago

thx

mumoshuover 6 years ago(edited)

I’m doubting something is wrong with your cdp-chart/templates/rbac.yaml. Could you share it?

Benover 6 years ago


{{- if .Values.rbac.create -}}
apiVersion: <http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>
kind: Role
metadata:
  name: {{ .Values.app.name }}
  namespace: {{ .Values.app.namespace }}
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  resourceNames:
  - "{{ .Values.app.name }}-config"
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - secrets
  resourceNames:
  - "{{ .Values.app.name }}-secret"
  - "gitlab-registry"
  - "{{ .Values.app.name }}-tls"
  verbs:
  - get
{{- end -}}
---
apiVersion: <http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>
kind: RoleBinding
metadata:
  name: {{ .Values.app.name }}
  namespace: {{ .Values.app.namespace }}
roleRef:
  apiGroup: <http://rbac.authorization.k8s.io|rbac.authorization.k8s.io>
  kind: Role
  name: {{ .Values.app.name }}
subjects:
- kind: ServiceAccount
  name: {{ .Values.app.name }}
  namespace: {{ .Values.app.namespace }}

mumoshuover 6 years ago

Also - enabling debug logs like helmfile --log-level=debug allows you to see the yaml after rendering the template, which would help debugging

Benover 6 years ago

Actually the error states the error occurs in serviceaccount.yaml which looks like


{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
  name: {{ .Values.app.name }}
  namespace: {{ .Values.app.namespace }}
  labels:
    app: {{ .Values.app.name }}
{{- end -}}

mumoshuover 6 years ago

ah sry I missed https://sweetops.slack.com/archives/CE5NGCB9Q/p1563859105017700

mumoshuover 6 years ago

indeed this can be a bug in helmfile!

mumoshuover 6 years ago

ahh it isn’t rendering nested strings.. i’ll open an issue for that

mumoshuover 6 years ago

@Ben Is this a blocker to you? (I’ll prioritize this accordingly if so

Benover 6 years ago

Thanks @mumoshu It's not a blocker but would definitely remove a lot of boilerplate config for us

Benover 6 years ago

Do you have any knowledge of how people are doing continuous deployment (or nearly CD with some manual gates before running sync) using helmfile? On the surface it seems simple; build, test, publish docker image, update helmfile with new image tag (or chart version if publishing a new chart) and run sync. However, it gets much more complicated once you have to deploy to multiple environments/tenants and also stage helmfile changes. Just wondering if you've seen any good approaches?

Erik Osterman (Cloud Posse)over 6 years ago

We run Helmfile under atlantis

Erik Osterman (Cloud Posse)over 6 years ago

We centralize our Helmfiles in a repo

Erik Osterman (Cloud Posse)over 6 years ago

Then use remote Helmfiles to pull them in pinned to a release (kind of like terraform)

Benover 6 years ago

Thanks @Erik Osterman (Cloud Posse) The thing I'm struggling with is how to update centralised helmfiles in an automated way. At the moment I can only see one option;
Each microservice git project's build pipeline creates/publishes docker image, clones the helmfile repo, creates a branch, using a script (sed, etc) updates the chart version or values, commits changes. Then another pipeline checks for helmfile repo changes and deploys to an environment based on branch name (feature/nnn, release/nnn, master). Once the helmfile sync is finished, a tester tests the system in a test env and if happy merges the helmfile repo branch into master. The same pipeline detects the change in git and deploys to prod because changes are now on master.
Does that seem reasonable? How does your process differ from this?

Andrew Nazarovover 6 years ago(edited)

@Ben We’ve got a single helmfile with all the releases for all the environments defined. It’s placed in a separate repo, not within the main codebase. Since we’ve got a set of stages (dev, several stagings, prod) and per client installations we are leveraging environments: feature extensively to make it DRY. The workflow is pretty manual indeed, one has to set a new version of the docker image and/or helm chart for the certain environment. To control things pull-request/merge-request workflow could be considered. Not the best approach for sure, but kinda work for us right now.

Andrew Nazarovover 6 years ago

@Erik Osterman (Cloud Posse) Out of curiosity, what is this Atlantis thing?

Micalover 6 years ago

Would be nice to have condition: boolean on hooks because sometimes you might want to run a hook only if installed: boolean is true on another release.. if course it can be wrapped by other logic but would be cleaner to have a flag for it

Micalover 6 years ago

bugs me that helm test --cleanup does not output the logs before removal on error 😐️

Emanuelover 6 years ago

Hiya. Currently I'm doing this:


releases:
- name: abc-namespace
  chart: stable/magic-namespace
  set:
  - name: tiller.image.tag
    value: v{{ .Values.tillerVersion }}
- name: xyz-namespace
  chart: stable/magic-namespace
  set:
  - name: tiller.image.tag
    value: v{{ .Values.tillerVersion }}

Is there a way to set tiller.image.tag as a default for all releases so that I don't have to specify that set every time? I've seen you can use release templates, but I wonder if there's an even more general way.

Micalover 6 years ago(edited)

can someone tell me if i'm not understanding environments: correctly and how it applies to releases. i have 2 files in helmfile.d:
istio.yaml


environments:
  staging:

releases:
  ...

app.yaml


environments:
  default:
  staging:

releases:
  ...

- if i run helmfile sync i expect only app.yaml releases to be synced
- if i run helmfile -e staging sync i expect both to be synced.

Micalover 6 years ago(edited)

don't want to wrap istio releases with {{ if eq .Environment.Name "staging" }}

Micalover 6 years ago

or is it that default applies to all implicitly?

Andrew Nazarovover 6 years ago

Basically you want to set values under envs and use it like:


environments:
  dev:
    values:
     - foo: aaa
  prod:
    values:
     - foo: bbb

releases:
  something: {{ .Environment.Values.foo }}

for helmfile -e dev apply something would be aaa
for helmfile -e prod apply something would be bbb

Micalover 6 years ago

yeah that i know, was wondering about the default environment if that is applied to all releases

Micalover 6 years ago(edited)

because if i replace default with dev in my case and run helmfile -e dev sync then it only applies to app.yaml releases

Micalover 6 years ago

i expect i was abusing the default 🙂

Micalover 6 years ago(edited)

but thanks for input @Andrew Nazarov (chose some bad names in my example btw.. updated)

Andrew Nazarovover 6 years ago

Actually, I thought environments don’t affect releases until you define something like mentioned {{ if eq .Environment.Name "app" }}. What you are talking about is something new to me. I mean, that helmfile -e dev syncs only releases in the app.yaml. It it documented somewhere?

Micalover 6 years ago

Not sure if it is, but it works that way at least 🙂

Andrew Nazarovover 6 years ago

I hope @mumoshu will comment.

Micalover 6 years ago(edited)

but i need {{ if ne .Environment.Name "default" }} around my istio releases to exclude them from the default env

Andrew Nazarovover 6 years ago(edited)

Ah, then it’s expected)). At first I thought you somehow managed to do it without if .Environment.Name since you wrote “don’t want to wrap istio releases ”

Andrew Nazarovover 6 years ago

Thanks for the clarification.

Micalover 6 years ago

I meant that i don't want to wrap it with {{ if eq .Environment.Name "xxx" }}

Micalover 6 years ago

not {{ if ne .... }}

Micalover 6 years ago

would prefer not having to add the not equals conditional though 🙂

Yannisover 6 years ago

Hey, just started to experiment with Helmfile and besides some hiccups with helm-diff, I was wondering, is it possible to capture directly stdout from a command and use it in a template? E.g. something like {{ requiredEnv "PLATFORM_ENV" }}, maybe {{ getOutput "echo Hello" }}?

Emanuelover 6 years ago

@Yannis is that not what {{ exec }} does? (I'm also new to Helmfile)

Emanuelover 6 years ago(edited)

https://github.com/roboll/helmfile/blob/master/README.md#importing-values-from-any-source

Yannisover 6 years ago

@Emanuel I must have been blind! Let's test it then 🙂

Emanuelover 6 years ago

I think I don't fully understand how environments work. If I have


helmfiles:
- ./*/helmfile.yaml

And I want to use helmfile -e production apply then does each of those subhelmfiles have to define the environment? Else you'd get err: no releases found that matches specified selector() and environment(production), in any helmfile

So what would that look like? Do I put this code at the top of each subhelmfile? There has to be a better way!


bases:
{{- range $_, $file := ( exec "sh" (list "-c" "echo ../environments/*yaml") | splitList " " ) }}
- {{ trim $file }}
{{ end }}

Emanuelover 6 years ago

If environments are global, I'd expect to define them in the parent helmfile only.

Micalover 6 years ago

A --no-hooks option to helmfile would be nice.

June 2019 Browse by date August 2019