78 messages
Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/
Balazs Vargaalmost 5 years ago
any idea why it does not work like values ?
My templates settings are :
skipping missing values file matching "../config/{{ .Release.Name }}/patches.yaml"
skipping missing values file matching "../config/{{ .Release.Name }}/{{ .Environment.Name }}-patches.yaml"
skipping missing values file matching "../config/{{ .Release.Name }}/merge.yaml"
skipping missing values file matching "../config/{{ .Release.Name }}/merge.yaml.gotmpl"
skipping missing values file matching "../config/{{ .Release.Name }}/{{ .Environment.Name }}-merge.yaml"
skipping missing values file matching "../config/app/values.yaml"
Successfully generated the value file at ../config/app/values.yaml.gotmpl. produced:My templates settings are :
templates:
default: &default
missingFileHandler: Debug
values:
- ../config/{{ .Release.Name }}/values.yaml
- ../config/{{ .Release.Name }}/values.yaml.gotmpl
- ../config/{{ .Release.Name }}/{{ .Environment.Name }}.yaml
- ../config/{{ .Release.Name }}/{{ .Environment.Name }}.yaml.gotmpl
secrets:
- ../config/{{ .Release.Name }}/secrets.yaml
- ../config/{{ .Release.Name }}/secrets.yaml.gotmpl
- ../config/{{ .Release.Name }}/{{ .Environment.Name }}-secrets.yaml
strategicMergePatches:
- ../config/{{ .Release.Name }}/merge.yaml
- ../config/{{ .Release.Name }}/merge.yaml.gotmpl
- ../config/{{ .Release.Name }}/{{ .Environment.Name }}-merge.yaml
jsonPatches:
- ../config/{{ .Release.Name }}/patches.yaml
- ../config/{{ .Release.Name }}/{{ .Environment.Name }}-patches.yamlPepperKickalmost 5 years ago(edited)
Hi everyone, this is probably me missing something simple but for some reason I am not able to use the new "waitForJobs" config.
I added it to the release section as follows
The
Helm and helmfile version
I added it to the release section as follows
releases:
- name: test
...
wait: true
waitForJobs: true
timeout: 60helmfiles:
...
{{ if eq .Environment.Name "cluster" }}
- path: environments/cluster/test.yaml
{{ end }}
environments:
cluster:The
helmfile apply command fails with...
[1] in /home/helmfile/helm-installer/resources/helmfile.yaml: in .helmfiles[1]: in environments/cluster/test.yaml: failed to read test.yaml: reading document at index 1: yaml: unmarshal errors:
[1] line 7: field waitForJobs not found in type state.ReleaseSpecHelm and helmfile version
bash-5.0# helm version
version.BuildInfo{Version:"v3.5.0", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"clean", GoVersion:"go1.15.6"}
bash-5.0# helmfile version
helmfile version v0.138.7mumoshualmost 5 years ago
Does anyone rely on helmfile's current behavior that
https://github.com/roboll/helmfile/pull/1772 can be a breaking change to you so please chime-in and leave your comments if you have opinions
helmfile -l name=foo apply NOT failing when foo had some needs to other releases?https://github.com/roboll/helmfile/pull/1772 can be a breaking change to you so please chime-in and leave your comments if you have opinions
Jim Conneralmost 5 years ago(edited)
Good evening everyone. I’m trying to learn
The
The
The
helmfile and am struggling to figure out why I’m getting the following errors. My directory structure is:.(helmfile.d)
├── generic
| ├── helmfile.yaml
│ ├── 01-secrets-management
│ │ ├── dex
│ │ │ ├── helmfile.yaml
│ │ │ └── values.yaml
│ │ ├── helmfile.yaml
│ │ ├── oauth2-proxy
│ │ │ ├── arm64-values.yaml
│ │ │ ├── secrets
│ │ │ ├── values.yaml
│ │ │ └── wait_for_endpoint.sh
│ │ ├── vault-operator
│ │ │ ├── helmfile.yaml
│ │ │ ├── secrets
│ │ │ │ └── vault-cr-secret-dec.yaml
│ │ │ └── values.yaml
│ │ └── vault-secrets-webhook
│ │ └── values.yaml
| └── common
│ ├── config.yaml
│ ├── environments.yaml
│ ├── helmdefaults.yaml
│ └── repos.yaml
└── helmfile.yamlThe
helmfile.yaml in . is:---
helmfiles:
- "*/*"The
helmfile.yaml in generic is:helmfiles:
- "*"The
helmfile.yaml in 01-secrets-management is:bases:
- ../common/environments.yaml
- ../common/repos.yaml
- ../common/helmdefaults.yamlJim Conneralmost 5 years ago
finally:
Jim Conneralmost 5 years ago(edited)
The
I’m running
helmfile.yaml in dex is:bases:
- ../../common/environments.yaml
- ../../common/repos.yaml
- ../../common/helmdefaults.yaml
## ************************************
## Start of DEX installation
## ************************************
releases:
- name: dex
namespace: {{ .Values.dex.namespace }}
createNamespace: true
labels:
tier: "secrets-management"
app: dex
chart: repo/helm-charts
version: {{ .Values.dex.version }}
values:
- values.yamlI’m running
helmfile -e default --log-level debug lint from the dex directory. I get the following output/error:J
Jim Conneralmost 5 years ago
Jim Conneralmost 5 years ago
The first error means that it’s not finding/using the common bases. I’m not understanding why. The last error is seemingly cascading from the first.
Jim Conneralmost 5 years ago
perhaps I’ve misread/misunderstood the documentation on
bases but re-reading it it seems that helmfile is trying to render helmfile.yaml BEFORE rendering the other layers. Is that correct?Jim Conneralmost 5 years ago
so if that’s the case then that means I can’t use anything defined in
environments because it’s not yet read/rendered. Correct?Jim Conneralmost 5 years ago
seems kinda useless to me if that’s the case.
Jim Conneralmost 5 years ago
FTR, if this is how
bases works, it seems that it’s not very useful in this case and would be more useful in top-level helmfile.yaml files.mumoshualmost 5 years ago
We can't resolve that chicken-and-egg problem automagically. I may be still missing something but at glance what you wanted seems like
mumoshualmost 5 years ago
bases:
- ../../common/environments.yaml
- ../../common/repos.yaml
- ../../common/helmdefaults.yaml
---
## ************************************
## Start of DEX installation
## ************************************
releases:
- name: dex
namespace: {{ .Values.dex.namespace }}
createNamespace: true
labels:
tier: "secrets-management"
app: dex
chart: repo/helm-charts
version: {{ .Values.dex.version }}
values:
- values.yamlmumoshualmost 5 years ago
Notice
--- so that the first part is rendered as a template to produce a YAML structure that includes bases. Bases should be loaded and the env values are loaded before rendering the latter part as a templateJim Conneralmost 5 years ago
ahh! Lemme give that a try real fast.
Jim Conneralmost 5 years ago
nice!
Jim Conneralmost 5 years ago
not outta the woods yet…but that got me past that roadblock
Jim Conneralmost 5 years ago
how much time you got @mumoshu? Is it late where you are? I have questions, thoughts, and possibly ideas.
Jim Conneralmost 5 years ago
and I’m still very new to helmfile so I’m struggling to get a good working set of helmfiles here.
J
Jim Conneralmost 5 years ago
The error has changed…but I think this is good as it seems that the
bases files are getting read now. These are the new errors.Jim Conneralmost 5 years ago
This looks like possibly over-lapping values between my
config.yaml or environments.yaml files.Jim Conneralmost 5 years ago
actually, this looks like it’s reading
environments.yaml which in turn references config.yaml within the directory where environments.yaml resides. This might be a referential problem with respect to … something.Jim Conneralmost 5 years ago
taking another look at https://github.com/roboll/helmfile#paths-overview
Jim Conneralmost 5 years ago
so
does this mean in sub-helmfile.yaml files that the path references in those files are relative to them or the relative to the top-level helmfile.yaml?
Relative paths referenced in the Helmfile manifest itself are relative to that manifest
does this mean in sub-helmfile.yaml files that the path references in those files are relative to them or the relative to the top-level helmfile.yaml?
Jim Conneralmost 5 years ago
I’m trying to use paths in sub-files as relative to the sub-files.
Jim Conneralmost 5 years ago
wow.
Jim Conneralmost 5 years ago
that’s the case.
Jim Conneralmost 5 years ago
@mumoshu that’s a super-confusing thing methinks.
Jim Conneralmost 5 years ago
it’s also not very portable imo.
Marianoalmost 5 years ago
hey guys, im trying to add helmfile to a docker image (dockerfile) and not sure if https://github.com/roboll/helmfile/releases/download/v0.138.7/helmfile_linux_amd64 is the actual binary?
Balazs Vargaalmost 5 years ago
can we set a "security" to avoid misdeploys? I mean if I accidentally deploy from a wrong branch to a cluster. ? a warning message or interactive promt... and not -i because we can forget it. in default config maybe?
Jim Conneralmost 5 years ago
I have the following directory structure where
In
The concept is that any specific configuration values for
I don’t understand why.
helmfile.d has the helmfile manifests. The helmfile directory tree has my environments and this is from where I want to invoke environmental helmfile runs:
├── helmfile
│ ├── envs
│ │ ├── dev
│ │ │ └── cluster-a
│ │ ├── preprod
│ │ └── production
│ └── shlib
└── helmfile.d
└── generic
├── 01a-network-and-proxies
│ ├── ambassador
│ ├── external-dns
│ └── ingress-nginx
├── 01b-secrets-management
│ ├── certmanager
│ ├── dex
│ ├── oauth2-proxy
│ │ └── secrets
│ ├── vault-operator
│ │ └── secrets
│ └── vault-secrets-webhook
└── commonIn
cluster-a there’s a helmfile.yaml and config.yaml:❯ \cat helmfile.yaml
environments:
default:
values:
- config.yaml
helmfiles:
- "../../../../helmfiles.d/*"The concept is that any specific configuration values for
cluster-a are specified in the config.yaml inside the cluster-a directory. However, when I test invoking helmfile from this path, I get the error:envvals_loader: loaded config.yaml:map[values:map[keyname:foobar]]
no matches for path: ../../../../helmfiles.d/*
merged environment: &{default map[values:map[keyname:foobar]] map[]}
helm:XVlBz> v3.4.1+gc4e7485
0 release(s) found in helmfile.yaml
err: no releases found that matches specified selector() and environment(default), in any helmfileI don’t understand why.
Jim Conneralmost 5 years ago(edited)
one thing to note is there’s a
helmfile.yaml in each directory which specifies helmfiles property:.(helmfile.d)
├── generic
│ ├── 01a-network-and-proxies
│ │ └── helmfile.yaml
│ ├── 01b-secrets-management
│ │ └── helmfile.yaml
│ └── helmfile.yaml
└── helmfile.yamlJim Conneralmost 5 years ago
in
helmfile.d/helmfile.yaml---
helmfiles:
- "generic/*"Jim Conneralmost 5 years ago
in
generic/helmfile.yamlhelmfiles:
- "*/*.yaml"Jim Conneralmost 5 years ago
won’t the attribute in
helmfiles: cause helmfile to “walk up the chain” so-to-speak?Jim Conneralmost 5 years ago
sigh, figured it out.
Jim Conneralmost 5 years ago
for posterity sake, the
helmfile.yaml in the cluster-a directory had a misspelled directory name: - "../../../../helmfiles.d/*" (note helmfiles.d vs helmfile.d)Jim Conneralmost 5 years ago
@mumoshu https://github.com/roboll/helmfile/issues/1045#issuecomment-820870785 might be interesting to you?
Jim Conneralmost 5 years ago
sigh. I’m really struggling to understand certain points in this documentation. I keep getting blocked on aspects of how to properly use
For instance, I’m blocked on certain aspects of this project (tickets and questions are submitted) and I’m moving on to other aspects of this project where I’m not blocked. One of those aspects is the ability to logically select charts. It doesn’t work the way I’m trying to implement so I’m reading the documentation, which doesn’t make sense to me. I don’t understand the context of inheritance with respect to sub-helmfiles and the inherited properties. That point is not well spelled out. What do you mean by inherited in context? Everything? Certain properties? The selectors? I don’t understand.
The use-case I’m trying to solve is to simply use a selector to run helmfiles only identified by that selector. Everything will run exactly the same as if I didn’t use any selector except ONLY the helmfiles specified by that selector (or a negated selector) would run. The details of how that works should not be something the end-user is concerned about if it’s clear-cut as what I was hoping would be the case. I’m simply not understanding, which I think is because the documentation is a little too sparse, but there is a lot of documentation. I can’t put my finger on the problem. It might be me, which I can concede. It shouldn’t be this complicated to understand what software can do and I keep running into roadblocks here. I want to use helmfile because I believe it does what we need it to do. Some assistance would be greatly appreciated.
helmfile and I either can’t find documentation to meet my needs or the documentation I think will help me either doesn’t make sense or is not as verbose as necessary to understand the full aspect of what the documentation is trying to point out.For instance, I’m blocked on certain aspects of this project (tickets and questions are submitted) and I’m moving on to other aspects of this project where I’m not blocked. One of those aspects is the ability to logically select charts. It doesn’t work the way I’m trying to implement so I’m reading the documentation, which doesn’t make sense to me. I don’t understand the context of inheritance with respect to sub-helmfiles and the inherited properties. That point is not well spelled out. What do you mean by inherited in context? Everything? Certain properties? The selectors? I don’t understand.
The use-case I’m trying to solve is to simply use a selector to run helmfiles only identified by that selector. Everything will run exactly the same as if I didn’t use any selector except ONLY the helmfiles specified by that selector (or a negated selector) would run. The details of how that works should not be something the end-user is concerned about if it’s clear-cut as what I was hoping would be the case. I’m simply not understanding, which I think is because the documentation is a little too sparse, but there is a lot of documentation. I can’t put my finger on the problem. It might be me, which I can concede. It shouldn’t be this complicated to understand what software can do and I keep running into roadblocks here. I want to use helmfile because I believe it does what we need it to do. Some assistance would be greatly appreciated.
jedineeperalmost 5 years ago
Do I have to explicitly define a reference to each of the values I include under a named environment block into a release or is there a tidier way to bulk include the contents of the environment block ontop of the release values?
Jim Conneralmost 5 years ago
@mumoshu we can continue the thread here
Alex Gencoalmost 5 years ago
Hello! I'm new to helmfile and need some guidance on a use case. I'm adding a nested helmfile inside a monorepo-like helmfile repository. The parent repo so far doesn't use environments since all services are global, but my child helmfile benefits greatly from them, because it defines production and staging namespaces and deployments. My helmfile looks basically like this:
This works great as long as I specify
I guess my question is, what's the best way to "inline" environments into the
However, it's attempting to "merge" the values files, rather than use the first to fill in the template of the second. Is there a better way to do this, rather than just duplicating the mostly-identical values files?
environments:
staging:
values:
- staging.yaml
production:
values:
- production.yaml
releases:
- name: app
chart: foo/app
namespace: app-{{.Environment.Name}}
values:
- values.yaml.gotmplThis works great as long as I specify
-e staging or -e production during helmfile apply. But since the parent repo has no concept of environments, it breaks the deploy process.I guess my question is, what's the best way to "inline" environments into the
releases list? i.e. I would like something like this:releases:
- name: app-staging
chart: foo/app
namespace: app-staging
values:
- staging.yaml
- values.yaml.gotmpl
- name: app-production
chart: foo/app
namespace: app-production
values:
- production.yaml
- values.yaml.gotmplHowever, it's attempting to "merge" the values files, rather than use the first to fill in the template of the second. Is there a better way to do this, rather than just duplicating the mostly-identical values files?
PePe Amengualalmost 5 years ago
Hi, I'm doing
hemlfile sync to change from a internet-facing alb to a internal alb and I tried destroy, sync and diff but even though there is changes the alb does not get get created, anyone had and issue like this before?Jim Conneralmost 5 years ago
@PePe Amengual it seems like it would be a better use-case for terraform ¯\(ツ)/¯ — Do you have some helmchart you’re using to deploy your ALB?
Jim Conneralmost 5 years ago
@Alex Genco I’d try to help you but I’m in the same boat as you man. Still learning.
Eugene Korekinalmost 5 years ago
is there any way to ignore helm errors for a specific release and continue?
Andrew Nazarovalmost 5 years ago
Victor Farcic spread the word about Helmfile on his channel: https://www.youtube.com/watch?v=qIJt8Iq8Zb0
muhahaalmost 5 years ago(edited)
guys, is there any replacement for deprecated inclubator/raw chart ?
Leia Renéealmost 5 years ago(edited)
Hi guys ,
Do you have a sample repository which installs kubernetes cluster auto scaler that works with TF 15.0 properly
I was using cookie labs which broken after upgrade.
Thanks
Leia
https://www.linkedin.com/in/leia-renee/
Do you have a sample repository which installs kubernetes cluster auto scaler that works with TF 15.0 properly
I was using cookie labs which broken after upgrade.
Thanks
Leia
https://www.linkedin.com/in/leia-renee/
Jim Conneralmost 5 years ago
@mumoshu I hope that helps more (the example I just gave in the issue. I’m heading to bed now…
Note that the example manifests from upstream (vault) show no single tics around the property. I added those for testing. With or without, it seems to be valid yaml. You can apply it successfully with
Note that the example manifests from upstream (vault) show no single tics around the property. I added those for testing. With or without, it seems to be valid yaml. You can apply it successfully with
kubectl but helmfile doesn’t process it right. Thanks for your help.Jim Conneralmost 5 years ago
gnight 👋🏼
mumoshualmost 5 years ago
I've curated a list of important feature requests and planned changes https://github.com/roboll/helmfile/issues/1806
Please feel free to review and add your voices if you find anything interesting to you 🙂
Please feel free to review and add your voices if you find anything interesting to you 🙂
esalmost 5 years ago
hi. is this a bug?
if I run
it fails with
if I run
helmfile lint against helmfile.yaml that containshelmfiles:
- environments/test1/helmfile.yaml
- environments/test2/helmfile3.yamlit fails with
Error: repo not found it’s referring to a release located in helmfile3.yaml. if I remove the first line (environments/test1/helmfile.yaml) it works fine. btw, I am using helmBinary option to specify helm version in each of those helmfilesesalmost 5 years ago
if I run
helmfile -f environments/test2/helmfile3.yaml repos it works. shouldn’t it automatically fetch charts?mumoshualmost 5 years ago
@es Does it work when you run
helmfile -f environments/test1/helmfile.yaml repos, too?mumoshualmost 5 years ago
Anyway, it would be great if you could provide provide us a more complete example for reproduction. At this point it's too hard to say if it's either a bug or not
Jim Conneralmost 5 years ago(edited)
@mumoshu I think I have a fundamental misunderstanding of how to use selectors for sub-helmfiles. You already have a basic skeleton of my repo/condition although I’ve made many changes to it so I’m gonna have to articulate my ask.
I’m using a layered approach to my helmfile project as you know. If you look in the
given:
if I want to be in
I’m using a layered approach to my helmfile project as you know. If you look in the
generic directory there is a 01a-… and 01b… directory for a tier of tools in each directory so I want to create a selector that would cause helmfile to 1) run everything in the respective 01a-… and 01b-… directories and 2) I want the flexibility to be able to just run one helmfile within a subdirectory of say, 01b-… (essentially just installing a single chart). It seems to me that the documentation says that the selectors are defined in the helmfiles directive of the parent directories to those sub-directories.given:
<project root>/helmfile.d/generic/01a-tier1 and <project root>/helmfile.d/generic/01b-tier2if I want to be in
helmfile.d to run helmfile and I want to install just charts in 01b-tier2 then the helmfile.yaml in helmfile.d would have to define the selectors and point to the helmfile.yaml in generic defining the selectors with a path: pointing to helmfile.yaml in 01b-tier2 and so forth, correct?Jim Conneralmost 5 years ago
My
My
helmfile.yaml in helmfile.d looks like:# helmfile.yaml in helmfile.d
---
helmfiles:
- "generic/*"
- path: "generic/common/*"My
helmfile.yaml in generic is:# helmfile.yaml in generic
---
helmfiles:
- "*/*.yaml"
- path: "common/*"
- path: "01a-network-and-proxies/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=network-and-proxies"
- path: "01b-secrets-management/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=secrets-managment"Jim Conneralmost 5 years ago
and finally, my
helmfile.yaml in 01b-… for instance is:# helmfile.yaml in security-management
---
helmfiles:
- "*/helmfile.yaml"
- path: "../common/*"
- path: "certmanager/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=secrets-managment"
- "app=certmanager"
- path: "vault-operator/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=secrets-managment"
- "app=vault-operator"
- path: "vault-secrets-webhook/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=secrets-managment"
- "app=vault-secrets-webhook"
- path: "dex/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=secrets-managment"
- "app=dex"
- path: "oauth2-proxy/helmfile.yaml"
values:
- {{ .Values | toYaml | nindent 8 }}
selectors:
- "tier=secrets-managment"
- "app=oauth2-proxy"Jim Conneralmost 5 years ago(edited)
I’m inclined to think that that is incorrect, though, because when I run
sync --selectors tier=secrets-managment,app=dex (or delete) for instance, it runs everything:J
Jim Conneralmost 5 years ago
mumoshualmost 5 years ago
Why are you giving
selectors: for every sub-helmfile? What if you just omitted that and let it install all when helmfile sync and let it install only part of releases with helmfile -l foo=bar sync?Jim Conneralmost 5 years ago
I guess because when I tested that, it didn’t work as I’d hoped…
Jim Conneralmost 5 years ago
maybe I did something wrong.
Jim Conneralmost 5 years ago
I was just looking at https://github.com/roboll/helmfile/blob/master/examples/README.md and I saw that helmfile is able to use the release labels…
Jim Conneralmost 5 years ago
so I’ll check that out again and see if I can find out why that doesn’t work.
Jim Conneralmost 5 years ago
nevertheless, I would like to be able to assign each full directory to a
tierJim Conneralmost 5 years ago
sorry for the slow response. I was at the store.
Jim Conneralmost 5 years ago
very cool. That seems to be working. Not sure what I was doing wrong before…
Jim Conneralmost 5 years ago
noice!
Jim Conneralmost 5 years ago
I have to go to the store again to get something for my wife but if you can answer this while I’m gone…
I thought I read in the docs once that there was a way to set a dependency up (like terraforms depends on) but of course, differently implemented. Did I misread that?
For instance, our implementation of dex requires a running vault. I’m sure there are ways to set up the environment that such a thing could be jiggered but is there a native way for helmfile to do this type of functionality?
I’ll be back in about an hour
I thought I read in the docs once that there was a way to set a dependency up (like terraforms depends on) but of course, differently implemented. Did I misread that?
For instance, our implementation of dex requires a running vault. I’m sure there are ways to set up the environment that such a thing could be jiggered but is there a native way for helmfile to do this type of functionality?
I’ll be back in about an hour
Jim Conneralmost 5 years ago
also, quick question about a thought on a feature request which may already exist. It would be super awesome if I could run an argument that would just show me the charts which would be affected by an operation on requested selector(s).
Jim Conneralmost 5 years ago
bbiab
Jim Conneralmost 5 years ago
cool. nm. don’t have to go now.
Jim Conneralmost 5 years ago
I thought I read in the docs once that there was a way to set a dependency up (like terraforms depends on) but of course, differently implemented. Did I misread that?
For instance, our implementation of dex requires a running vault. I’m sure there are ways to set up the environment that such a thing could be jiggered but is there a native way for helmfile to do this type of functionality?
Jim Conneralmost 5 years ago(edited)
@mumoshu if you’re available: how on earth would one define globally used values for all releases in an environment. This has been my absolute biggest blocker. Even better is that the global scoped values.yaml could be also templatized. I cannot figure out how to achieve this goal. Nothing I’ve tried works.