#helmfile - May 2019 | Slack Archive

113 messages

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/

Apr 2019

May 2019

Jun 2019

SeBalmost 7 years ago

hello there,
I am launching an helmfile apply on this

releases: 
- name: redis
  chart: stable/redis
  hooks:
    - events: ["cleanup"]
      command: "echo"
      args: ["foo bar"]

And expect to see “foo bar” on my console but nothing appears ?

SeBalmost 7 years ago

what am I doing wrong ?

mumoshualmost 7 years ago(edited)

Do you guys want the new config bases: ["mybase.yaml"] added to helmfile, even though it won’t support templating, and yaml anchors across the border? https://github.com/roboll/helmfile/issues/388#issuecomment-489633839

mumoshualmost 7 years ago

Started to think we should require .gotpl extension to enable the whole-file templating of helmfile.yaml https://github.com/roboll/helmfile/pull/573#issuecomment-491142465

@SeB @Erik Osterman (Cloud Posse) I’m interested in your opinion!

mumoshualmost 7 years ago

Also cc/ @Shane

mumoshualmost 7 years ago

https://github.com/roboll/helmfile/issues/584

Shanealmost 7 years ago

Wait I thought we currently required a extension

Shanealmost 7 years ago

Yaml files don't get templates

Shanealmost 7 years ago

What am I missing

mumoshualmost 7 years ago

@Shane Yep, for values.yaml that’s true

mumoshualmost 7 years ago

I’m proposing the same for helmfile.yaml

Shanealmost 7 years ago

Ahh

mumoshualmost 7 years ago

(I think we’ve discussed about that long before… but at that time we thought it’s better to just use .yaml as helm templates do the same

Shanealmost 7 years ago

I typically don't put anything in my helmfile except for chart references

mumoshualmost 7 years ago

Oh!

Shanealmost 7 years ago

Hence never thinking about the templating of helmfile

mumoshualmost 7 years ago

Do you use another tools to make it work against various envs, repositories, and so on?

Shanealmost 7 years ago

I keep pushing our teams to leverage it as little as possible if we control the chart

mumoshualmost 7 years ago

i.e. generating helmfile.yaml with a tool like jsonnet

Shanealmost 7 years ago

All values are kept in separate values files

Shanealmost 7 years ago

So helmfile is a list of charts and value references

mumoshualmost 7 years ago

that’s a neat idea 🙂

mumoshualmost 7 years ago

so you maintain one helmfile.yaml per environment

Shanealmost 7 years ago

Almost yes

Shanealmost 7 years ago

In dev we have a helmfile per team also

mumoshualmost 7 years ago

makes sense!

mumoshualmost 7 years ago(edited)

i think the helmfile.yaml template is used (1)when people want to create a single helmfile.yaml that is the entrypoint to all the flavors of environments or (2)make each helmfile.yaml DRY when one has to maintain many of them, without introducing another tools like jsonnet

mumoshualmost 7 years ago

so use-case like yours completely make sense to me

Shanealmost 7 years ago

Ya tons of options

SeBalmost 7 years ago

Hello, @mumoshu
1) could you explain in detail why we can’t consider helmfile.yaml as a template file and why we need to have both extensions.
2) Also why not using the same extension as values (gotmpl vs gotpl) or vise-versa otherwise this will be confusing.

mumoshualmost 7 years ago(edited)

For 1), it doesn’t work well with release templating.

mumoshualmost 7 years ago

For 2), gotmpl and gotpl should be merged. I think I meant gotmpl 🙂

mumoshualmost 7 years ago

An example related to 1) can be seen at https://github.com/roboll/helmfile/pull/573

mumoshualmost 7 years ago


releases:
  - name: redis-test
    chart: stable/redis
    values:
    - fake_key:
        redis_pass_east: {{ ssm "east1:redis_password" }}

{{ ssm "east1:redis_password" }} is evaluated twice in the above example.

mumoshualmost 7 years ago

Another example for release templates, is that you can’t write:


releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

It must be:


releases:
  - name: redis-test
    values:
    - releases/{{` {{ .Release.Name }} `}}/values.yaml

Due to that helmfile.yaml itself is a go template.

SeBalmost 7 years ago(edited)

thank @mumoshu for your quick answer, but change the extension to gotmpl whould change what in the above examples ?

mumoshualmost 7 years ago

@SeB What I suggested was to “add” .gotmpl ext. to explicitly enable helmfile.yaml templating. And release templates are always enabled. That being said

SeBalmost 7 years ago

but why not consider always helmfile.yaml a templates ?

mumoshualmost 7 years ago

Making helmfile require .gotmpl to enable helmfile.yaml templating allows you to write a regular YAML file named helmfile.yaml

mumoshualmost 7 years ago

So that


releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

THis is a valid helmfile.yaml with just release templaets

mumoshualmost 7 years ago

Note that it isn’t a valid helmfile.yaml if helmfile.yaml itself is a go template

mumoshualmost 7 years ago

This happens because there’s two template contexts. When templating helmfile.yaml as a whole, you have no access to {{ .Release }}

mumoshualmost 7 years ago

When templating helmfile.yaml as a whole, we have no {{ .Release }} in the template context available, because helmfile.yaml is basically a template that “produces YAML” .
No way to know what releases are defined in the helmfile.yaml before rendering and parsing it as YAML…

SeBalmost 7 years ago

ok, I think I got it

mumoshualmost 7 years ago(edited)

And no way to know which {{ .Release }} it is rendering while templating helmfile.yaml as a whole

mumoshualmost 7 years ago

// {{ .Release }} becomes available only when helmfile iterates over each release defined in the YAML

SeBalmost 7 years ago

so what you propose is to have 2 extensions :
1) helmfile.yaml

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

2) helmfie.gotmpl

releases:
  - name: redis-test
    values:
    - releases/{{` {{ .Release.Name }} `}}/values.yaml

SeBalmost 7 years ago

to have the same result

mumoshualmost 7 years ago

Exactly!

SeBalmost 7 years ago

I am afraid this is too complicated for the end-user.

mumoshualmost 7 years ago

Yep it is

mumoshualmost 7 years ago

I’m thinking that it is already complicated and we’d better migrate towards explicit file ext. towards helmfile v1

mumoshualmost 7 years ago

so that it becomes simpler

SeBalmost 7 years ago

I beleive this could be just much better explained in the documentation, this is what it is lacking today. The of precise documentation make pleople to hack/try/fail to have their helmfile working

mumoshualmost 7 years ago

For advanced users I’d still suggest using helmfile.gotmpl. But as I’ve explained in the issue, you can fully achieve the same result even with helmfile.yaml without gotmpl, when combined with bases

mumoshualmost 7 years ago

Hm, maybe in that case I completely remove helmfile.yaml and only helmfile.gotmpl supported.

mumoshualmost 7 years ago

so that there’s no confusion anyway.

mumoshualmost 7 years ago(edited)

And the docs you’ve suggested would help people understand why one needs backticks sometimes

SeBalmost 7 years ago

yep

mumoshualmost 7 years ago

Great! Thanks for all the feedbacks

mumoshualmost 7 years ago

So we can proceed that way

mumoshualmost 7 years ago

And discuss about re-introducing helmfile.yaml but as a regular YAML file this time separately in longer term, maybe after docs improvements, if it is really people want

mumoshualmost 7 years ago

Summarized our discussion in https://github.com/roboll/helmfile/issues/584#issuecomment-491177956

mumoshualmost 7 years ago(edited)

It is interesting that helmfile.yaml, rather than helmfile.gotmpl, started to look like for advanced users 🙂

Ufoualmost 7 years ago

is anyone successfully using helmfile with keel?

mumoshualmost 7 years ago(edited)

@Ufou Hey! Unfortunately I have never tried that myself. And I think you’d need to enhance keelhq a bit to “fully” make it work with helmfile

mumoshualmost 7 years ago

Keel updates your container images referenced by your apps by calling helm on its own.

I thought what you want would be keel calls helmfile to update container image tags of your apps. That’s not how keel works at the moment

mumoshualmost 7 years ago

Anyway, in case you really want keel just for updating your container image tags out of helmfile, just add keel-related values to your values.yaml managed by helmfile.

https://keel.sh/v1/guide/documentation.html#Helm-example

Ufoualmost 7 years ago

@mumoshu thanks! Yeah, I worked out as much, I just didn’t get it working yet and wondered if others were successful with it…

mumoshualmost 7 years ago

@Shane I hope you like this

https://twitter.com/mumoshu/status/1128218333315100673

Andrew Nazarovalmost 7 years ago

@mumoshu are you gonna make helmfile leverage helm-x to work with plain manifests or kustomize’ed stuff? I’m joking, but have you ever had these thoughts - to manage not only helm via a tool like helmfile. Actually, once I had a request about how to provide management of objects with one tool while some services were helmified, but others were not migrated yet.

Shanealmost 7 years ago

@mumoshu keep up the good work 🙂

Shanealmost 7 years ago

@mumoshu intriguing

Running helmfile from helm/account/dev


in ./helmfile.yaml: in .helmfiles[0]: in helm/account/helmfile.yaml: failed to read helmfile.yaml: helm exited with status 1:
  error loading config: no matching creation rules found
  Error: plugin "secrets" exited with error
 less ./helmfile.yaml                                                                                                                             
 less helm/account/helmfile.yaml                                                                                                                    
 ls -larth helm/account/helmfile.yaml                                                                                                               
-rw-r--r--  1 sstarcher  staff   4.5K May  8 17:50 helm/account/helmfile.yaml
 ls -larth ./helmfile.yaml                                                                                                                          
-rw-r--r--  1 sstarcher  staff   422B May 13 15:51 ./helmfile.yaml

Shanealmost 7 years ago

so it's the secrets plugin choking

Shanealmost 7 years ago

Has something changed with the directory structure. It is not longer finding my .sops.yaml for sub helm charts.

Shanealmost 7 years ago

nope... something on my end has had to of changed

Shanealmost 7 years ago

So ya, probably certainly seems to be with secret decryption, but I can't find out what would of changed. My .sops.yaml has always been at the root. Did it by chance change to run the secret decryption rooted from the helmfile instead of the root helmfile?

Shanealmost 7 years ago

Ya, seems something has changed related to sops. I now require a .sops.yaml in each of my directories that have a helmfile.

Shanealmost 7 years ago

Sops had a release 26 days ago so I also changed to the older sops release 3.2.0 from 2018. Both seem to have the same issue with helmfile.

Shanealmost 7 years ago

Well... I can fix it by changing ../../_environment/secrets.yaml to ../../../dev/_environment/secrets.yaml, but that does not explain why it just started failing.

Shanealmost 7 years ago

well I think I find a viable workaround... but then I caught things on even worse fire

Shanealmost 7 years ago

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x13cb127]

goroutine 1 [running]:
<http://github.com/urfave/cli.HandleAction.func1(0xc000251000)|github.com/urfave/cli.HandleAction.func1(0xc000251000)>
	/go/pkg/mod/github.com/urfave/cli@v0.0.0-20160620154522-6011f165dc28/app.go:474 +0x287
panic(0x145a340, 0x186a7a0)
	/usr/local/go/src/runtime/panic.go:522 +0x1b5
<http://github.com/roboll/helmfile/pkg/app.(*Error).Error|github.com/roboll/helmfile/pkg/app.(*Error).Error>(0xc000192000, 0xc000192060, 0x2)
	/home/circleci/workspace/helmfile/pkg/app/app.go:343 +0xb7
<http://github.com/roboll/helmfile/pkg/app.(*Error).Error|github.com/roboll/helmfile/pkg/app.(*Error).Error>(0xc000192030, 0xc000194090, 0x10ce2ee)
	/home/circleci/workspace/helmfile/pkg/app/app.go:339 +0x494
<http://github.com/roboll/helmfile/pkg/app.(*Error).Error(0xc000192060|github.com/roboll/helmfile/pkg/app.(*Error).Error(0xc000192060>, 0x0, 0x12)
	/home/circleci/workspace/helmfile/pkg/app/app.go:339 +0x494
<http://github.com/roboll/helmfile/pkg/app.(*Error).Error|github.com/roboll/helmfile/pkg/app.(*Error).Error>(0xc000192090, 0xc00016c090, 0x0)
	/home/circleci/workspace/helmfile/pkg/app/app.go:339 +0x494
<http://github.com/roboll/helmfile/cmd.toCliError|github.com/roboll/helmfile/cmd.toCliError>(0x1567d20, 0xc000192090, 0x0, 0xc00000cc40)
	/home/circleci/workspace/helmfile/cmd/cmd.go:88 +0xa7
<http://github.com/roboll/helmfile/cmd.FindAndIterateOverDesiredStatesUsingFlagsWithReverse(0xc000184140|github.com/roboll/helmfile/cmd.FindAndIterateOverDesiredStatesUsingFlagsWithReverse(0xc000184140>, 0xc00005e500, 0xc00005e5c0, 0x8, 0xc00000cbe0)
	/home/circleci/workspace/helmfile/cmd/cmd.go:79 +0xfa
main.findAndIterateOverDesiredStatesUsingFlags(...)
	/home/circleci/workspace/helmfile/main.go:619
main.main.func4(0xc000184140, 0x0, 0x0)
	/home/circleci/workspace/helmfile/main.go:220 +0x66
reflect.Value.call(0x143e300, 0x14e5af0, 0x13, 0x14c82e4, 0x4, 0xc000132fa0, 0x1, 0x1, 0xc00002e080, 0x1011bc3, ...)
	/usr/local/go/src/reflect/value.go:447 +0x461
reflect.Value.Call(0x143e300, 0x14e5af0, 0x13, 0xc000132fa0, 0x1, 0x1, 0xc00008a820, 0xc00008a868, 0x140)
	/usr/local/go/src/reflect/value.go:308 +0xa4
<http://github.com/urfave/cli.HandleAction(0x143e300|github.com/urfave/cli.HandleAction(0x143e300>, 0x14e5af0, 0xc000184140, 0x0, 0x0)
	/go/pkg/mod/github.com/urfave/cli@v0.0.0-20160620154522-6011f165dc28/app.go:483 +0x1ff
<http://github.com/urfave/cli.Command.Run(0x14c994c|github.com/urfave/cli.Command.Run(0x14c994c>, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x14de166, 0x3d, 0x0, ...)
	/go/pkg/mod/github.com/urfave/cli@v0.0.0-20160620154522-6011f165dc28/command.go:186 +0x8d1
<http://github.com/urfave/cli.(*App).Run(0xc000000f00|github.com/urfave/cli.(*App).Run(0xc000000f00>, 0xc0000a2000, 0x5, 0x5, 0x0, 0x0)
	/go/pkg/mod/github.com/urfave/cli@v0.0.0-20160620154522-6011f165dc28/app.go:237 +0x601
main.main()
	/home/circleci/workspace/helmfile/main.go:578 +0x1fef

Shanealmost 7 years ago

helmfile version v0.60.1

Shanealmost 7 years ago

well fixed the 2 errors i was getting - https://github.com/roboll/helmfile/pull/594

Andrew Nazarovalmost 7 years ago

@mumoshu sounds great!

erikalmost 7 years ago

set the channel topic: https://github.com/roboll/helmfile

Shanealmost 7 years ago

@mumoshu seeing some side effects from the new lockfile addition - https://github.com/roboll/helmfile/issues/598

mumoshualmost 7 years ago(edited)

While my trial on replicated/ship, I came up with how we can potentially leverage kustomize to patch helm releases, but still installing it as a helm chart.

https://github.com/mumoshu/helm-x/issues/2

I think we can extend this to helmfile, so that helmfile can be used for applying adhoc “patches” to a helm chart without forking. The patches can be used for injecting init/sidecar containers, modifying other pod spec, adding configmap/secret required but not included in the chart, etc…

Andrew Nazarovalmost 7 years ago

Btw, the latest version of helmfile available via homebrew is v0.54.2.

Collinalmost 7 years ago(edited)

@Andrew Nazarov @mumoshu Was just coming online to ping you about the homebrew version 😄 Are we going to see an update for it in the future? I'm asking before I manually install it from git and there are some breaking changes that make my life hell.

Collinalmost 7 years ago(edited)

And a side (noobish) question: We're trying to update some of our config maps via helm upgrade commands using --set but we're unable to target the chart as it's part of our helmfile. I'd normally use sync but I want to try a few direct changes. Am I being an idiot or would we have to move to individual chart files to be able to do this?

Pierre Humberdrozover 6 years ago(edited)

Hey,

How can I manage deployments of crd’s with helmfile? 🙂
I am currently migrating parts of our infra to helmfile stored in a repo and I need to deploy cert-manager as well

btaiover 6 years ago

does helmfile remove the need to manually bump chart versions etc in the repo?

Andriy Knysh (Cloud Posse)over 6 years ago

you have to specify the chart version directly in the helmfile, either by hardcoding it (e.g. https://github.com/cloudposse/helmfiles/blob/master/releases/cert-manager.yaml#L30) , or using ENV var

btaiover 6 years ago

thanks @Andriy Knysh (Cloud Posse)

Erik Osterman (Cloud Posse)over 6 years ago

@btai here's an example of using the raw chart

Erik Osterman (Cloud Posse)over 6 years ago

https://github.com/cloudposse/helmfiles/blob/master/releases/external-storage.yaml

Erik Osterman (Cloud Posse)over 6 years ago

e.g. if you don't have the chart for the CRDs, but still want to use helmfile

Ufouover 6 years ago

good morning, is it possible for helmfile to merge a map coming from environment yaml files so that I can take 2 maps and merge them together? eg if my values file looks something like:

default_config:
  resources:
    limits:
      cpu: 0.1
      memory: 200Mi
    requests:
      cpu: 0.1
      memory: 200Mi

service_config_a:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
  resources:
    limits:
      cpu: 0.3
      memory: 400Mi
    requests:
      cpu: 0.3
      memory: 400Mi

service_config_b:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70

Ufouover 6 years ago

I’d like service_config_b to inherit values from default_config

mumoshuover 6 years ago

Does using YAML anchor work?


default_config: &default_config
  resources:
    limits:
      cpu: 0.1
      memory: 200Mi
    requests:
      cpu: 0.1
      memory: 200Mi

service_config_a:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
  resources:
    limits:
      cpu: 0.3
      memory: 400Mi
    requests:
      cpu: 0.3
      memory: 400Mi

service_config_b:
  <<: *default_config
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70

Ufouover 6 years ago

hmm, I did not consider that, it may well do!

Ufouover 6 years ago

@mumoshu lovely stuff, thanks works beautifully

Ufouover 6 years ago

one other question - is the work for bases released yet?

mumoshuover 6 years ago

@Ufou yep! https://github.com/roboll/helmfile/releases/tag/v0.68.0 is the latest release including the bases feature, which all the known issues related to bases fixed

Ufouover 6 years ago

thanks!

Andrew Nazarovover 6 years ago

@mumoshu what is the release schedule for helmfile if there is one?

Andrew Nazarovover 6 years ago(edited)

I’m gonna reorganise things related to helmfile, try new features and stuff. @mumoshu Should I avoid set in my helmfile.yaml? I noticed somewhere you stated, that it was added for backward compatibility.

Erik Osterman (Cloud Posse)over 6 years ago

anyone see how far they could get with helm3-alpha?

Erik Osterman (Cloud Posse)over 6 years ago

https://github.com/helm/helm/releases/tag/v3.0.0-alpha.1

Andrew Nazarovover 6 years ago

Can I define several templates? Something like


templates:
  - first: &first
    snip1
  - second: &second
    snip2

Andrew Nazarovover 6 years ago(edited)

Why helm-tiller is not bundled in a helmfile’s Docker image?

mumoshuover 6 years ago

I was just lazy to do so! PR welcomed 🙂

April 2019 Browse by date June 2019