#helmfile - July 2020 | Slack Archive

41 messages

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/

Jun 2020

July 2020

Aug 2020

opollieover 5 years ago

hi all,
i am pretty much new to helmfile and currently trying to migrate a huge helm2 umbrella chart to helmfile...
one of my problems is for some reason the needs: concept doesn't work in the way it's written in the doc (or at least i got it wrong):
when i do sync this:

- name: loggingMaster
  chart: elastic/elasticsearch
  version: 7.3.0
  condition: loggingMaster.enabled
  <<: *default
- name: loggingData
  chart: elastic/elasticsearch
  version: 7.3.0
  condition: loggingData.enabled
  <<: *default
- name: misc-es
  chart: ./charts/misc-es
  version: 0.1.0
  verify: false
  condition: loggingMaster.enabled
  needs:
  - loggingMaster

i get the following error:

in ./helmfile.yaml: "clusterinfra/misc-es" depends on nonexistent release "loggingMaster"

any idea why?

soumyaover 5 years ago

Is there a way I can prevent creation of new config maps every time I deploy through helm.

seanover 5 years ago

Hi guys i'm having trouble getting readFile working with requiredEnv templating.

I currently have dev.yaml

environments:
  dev:
    values:
      - HelmRepo: custom-staging
{{ readFile "./helmfile-cs-base.yaml" }}

and helmfile-cs-base.yaml

<snip>

repositories:
  - name: custom
    url: <https://chartmuseum.internal.ourdomain.com/master>
    certFile: {{ requiredEnv CHARTMUSEUM_CERT_FILE }}
    keyFile: {{ requiredEnv CHARTMUSEUM_KEY_FILE }}
  - name: custom-staging
    url: <https://chartmuseum.internal.ourdomain.com/staging>
    certFile: {{ requiredEnv CHARTMUSEUM_CERT_FILE }}
    keyFile: {{ requiredEnv CHARTMUSEUM_KEY_FILE }}
    
<snip>

And when running it i get

err: failed to read helmfile-cs.yaml: reading document at index 1: yaml: unmarshal errors:
  line 32: cannot unmarshal !!map into string
  line 33: cannot unmarshal !!map into string
  line 36: cannot unmarshal !!map into string
  line 37: cannot unmarshal !!map into string
in ./helmfile-cs.yaml: failed to read helmfile-cs.yaml: reading document at index 1: yaml: unmarshal errors:
  line 32: cannot unmarshal !!map into string
  line 33: cannot unmarshal !!map into string
  line 36: cannot unmarshal !!map into string
  line 37: cannot unmarshal !!map into string

which corresponds to the repositories chunk.

im running helmfile version v0.119.1

Has anyone got any ideas?

jason witkowskiover 5 years ago(edited)

Hey All, I'm having a really confusing problem that I think someone better than I can maybe easily pinpoint. I have a straight-forward helm file. I pull in a values file to the environment and try to use a top level variable and I immediately get an error.
helmfile:

environments:
  preprod:
    values:
      - vars/helmfile/realms/preprod.yaml

releases:
{{- $realm := .Values.realm -}}
...

values-file:

$ cat vars/helmfile/realms/preprod.yaml
realm: preprod

Error:

$ helmfile -e preprod -f helmfile-node-local-dns.yaml lint
in ./helmfile-node-local-dns.yaml: error during helmfile-node-local-dns.yaml.part.0 parsing: template: stringTemplate:18:21: executing "stringTemplate" at <.Values.realm>: map has no entry for key "realm"

jason witkowskiover 5 years ago

Can anyone see what I'm doing wrong?

seanover 5 years ago

Hi guys. I'm trying to figure out how to override the environment values in a helmfile from the CLI and i'm not sure how to go about it, since I think the cli flag provided values --state-values-set ImageTag=latest-master are being overriden by the environment values specified in my helmfile.

i'm running

helmfile -f dev.yaml --debug -e flaky-smalt-cat-aujzyfqd0cc --interactive  --state-values-set ImageTag=latest-master apply --context=3

Is there anyway to override what ImageTag is in dev.yaml from the CLI?
dev.yaml

environments:
  flaky-smalt-cat-aujzyfqd0cc:
    values:
    - HelmRepo: company-staging
    - Namespace: flaky-smalt-cat-aujzyfqd0cc
    - ImageTag: latest-staging
    - CustomerTag: flaky-smalt-cat-aujzyfqd0cc
    - Monolith: true
    - Domain: company.dev

bases:
  - ./namespace.yaml

namespace.yaml

seanover 5 years ago

environments:
  default:
    values:
      - HelmRepo: company
      - Namespace: default
      - ImageTag: latest
      - CustomerTag: default-customer-tag

templates:

  chartnamespace: &chartnamespace
    namespace: "{{`{{ .Environment.Values.Namespace }}`}}"
    # missingFileHandler: Warn

  set: &set
    setTemplate:
      - name: image_tag
        value: "{{`{{ .Environment.Values.ImageTag }}`}}"
      - name: customerTag
        value: "{{`{{ .Environment.Values.CustomerTag }}`}}"
      - name: namespace
        value: "{{`{{ .Environment.Values.Namespace }}`}}"
      - name: baseHost
        value: "{{`.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"

  cs-ui-default: &cs-ui-default
    <<: *chartnamespace
    setTemplate:
      - name: image_tag
        value: "{{`{{ .Environment.Values.ImageTag }}`}}"
      - name: customerTag
        value: "{{`{{ .Environment.Values.CustomerTag }}`}}"
      - name: namespace
        value: "{{`{{ .Environment.Values.Namespace }}`}}"
      - name: config.server_name
        value: "{{`{{ .Environment.Values.CustomerTag }}.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"
      - name: baseHost
        value: "{{`.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"

  default: &default
    <<: *set
    <<: *chartnamespace

  cs-engine: &cs-engine
    name: cs-engine{{`{{ if ne .Environment.Values.Namespace "default" }}-{{ .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-engine"
    <<: *chartnamespace
    setTemplate:
      - name: image_tag
        value: "{{`{{ .Environment.Values.ImageTag }}`}}"
      - name: customerTag
        value: "{{`{{ .Environment.Values.CustomerTag }}`}}"
      - name: namespace
        value: "{{`{{ .Environment.Values.Namespace }}`}}"
      - name: baseHost
        value: "{{`{{ .Environment.Values.CustomerTag }}.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"
      - name: config.server_name
        value: "{{`{{ .Environment.Values.CustomerTag }}.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"
      - name: monolith.enabled
        value: "{{`{{ .Environment.Values | get \"Monolith\" \"false\" }}`}}"
      - name: baseHost
        value: "{{`.{{ .Environment.Values | get \"Domain\" \"<http://companycloud.com|companycloud.com>\" }}`}}"

  cs-ui: &cs-ui
    name: cs-ui{{`{{ if ne .Environment.Values.Namespace "default" }}-{{  .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-ui"
    <<: *cs-ui-default
  cs-api: &cs-api
    name: cs-api{{`{{ if ne .Environment.Values.Namespace "default" }}-{{  .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-api"
    <<: *default
  cs-database: &cs-database
    name: cs-database{{`{{ if ne .Environment.Values.Namespace "default" }}-{{  .Environment.Values.Namespace }}{{ end }}`}}
    chart: "{{`{{ .Environment.Values.HelmRepo }}`}}/cs-database"
    <<: *default

repositories:
  - name: company
    url: <https://chartmuseum.companycloud.com/master>
    certFile: {{ requiredEnv "CHARTMUSEUM_CERT_FILE"  }}
    keyFile: {{ requiredEnv "CHARTMUSEUM_KEY_FILE" }}
  - name: company-staging
    url: <https://chartmuseum.companycloud.com/staging>
    certFile: {{ requiredEnv "CHARTMUSEUM_CERT_FILE" }}
    keyFile: {{ requiredEnv "CHARTMUSEUM_KEY_FILE" }}

helmDefaults:
  timeout: 600
  recreatePods: true
  atomic: true
  force: true

releases:
  - <<: *cs-engine
  - <<: *cs-ui
  - <<: *cs-api
  - <<: *cs-database

muhahaover 5 years ago(edited)

guys? how are you installing cert-manager? this https://github.com/cloudposse/helmfiles/blob/master/releases/cert-manager.yaml will not remove CRDs in helmfile delete phase.

bradymover 5 years ago(edited)

I have a repo for which I need to deploy multiple releases: one is the main app, the other is a worker. I want them as separate releases as the worker should be able to scale independently of the main app. I have this part working great with helmfile. The worker listens to an SQS queue and processes the messages as they come in.

In stage I create an environment every time a new branch is deployed, and tear it down when that branch is merged. I'm trying to figure out the best way to specify when the worker should be deployed. Ideally we'd control that via an aws ssm parameter as that's how we manage our env vars. I was hoping I could do something like:

releases:
  - name: worker
    installed: <secretref+awsssm://V1/{{> .Values.repo }}/{{ .Environment.Name }}/DEPLOY_WORKER?region=us-west-1

But that doesn't work. Any ideas on how I could handle this? I'd like to avoid having a separate SQS queue per branch to keep down the number of things that have to be created/torn down.

Jonathanover 5 years ago

Been trying to access secrets from GCP Secret manager the following way in my values file:

password: <ref+gcpsecrets://project/secret-name?version=1>

And using a service account, which I can access the secret manually.
Despite this, I get an error regarding accessing the secret:

failed to render values files "config/values.yaml": expand <gcpsecrets://project/secret-name?version=1>: failed to get secret: rpc error: code = PermissionDenied desc = Request had insufficient authentication scopes.

It seems the service account permissions are bad, but they work when accessing manually:

bash-4.3# gcloud auth activate-service-account --key-file=keyfile.json
Activated service account credentials for: [<SERVICE_ACCOUNT>]
bash-4.3# gcloud secrets versions access 1 --secret="secret-name"

Does anyone have any pointers?

muhahaover 5 years ago

How can I override environment values in helmfile? For example:

environments:
  default:
    missingFileHandler: Warn
    values:
       - external_dns:
           identity:
             tenant_id: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_TENANT_ID" }}
             subscription_id: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_SUBSCRIPTION_ID" }}
             resource_group: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_RESOURCE_GROUP" }}
             name: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_NAME" }}
             id: {{ requiredEnv "EXTERNAL_DNS_IDENTITY_ID" }}
       - values.external-dns.yaml

It should fail if values.external-dns.yaml is not present/populated && env vars are not supplied.

Craig Dunfordover 5 years ago

We are looking at storing our helm charts in an Azure ACR. Per https://docs.microsoft.com/en-us/azure/container-registry/container-registry-helm-repos the recommendation is to use helm's experimental OCI support to publish the charts to the registry. Does anyone know if helmfile supports pulling charts from an OCI repository via helm's experimental support?

vgdubreaover 5 years ago

Hi Guys , I have a short question , any suggestions would be really helpful .

---
bases:
  - helmfile-first.yaml
  - helmfile-second.yaml

I would like to conditionally execute the helmfile-second.yaml without using if , else . is there an easy way I can do it ?

vgdubreaover 5 years ago

my end goal is to avoid any releases mentioned in second Helmfile to be deployed.

muhahaover 5 years ago

Guys? How should I version dynamic variables used in k8s deployments created with terraform ? Should I back commit these into git and then use helmfile with these values ? ( these are not secrets, just dynamic vars, not predictable )

Abhishekover 5 years ago

Hi everyone ! I have question, In case of nested helmfiles, do children helmfile inherit environment variables from parent helmfile. I'm not able to get the values in child helmfile. Below is my master helmfile and I'm trying to access values from imported-values.yaml but it's getting values only from folder/values.yaml

environments:
  default:
    values:
      - "imported-values.yaml"

helmfiles:
  - path: folder/helmfile.yaml
    values:
      - "folder/values.yaml"

Marcin Brańskiover 5 years ago

I’m kinda new to helmfile and haven’t yet figured everything out.
I’d like to use environment variable inside default function

tags: '{{ env "KUBE_PROMETHEUS_ALERT_MANAGER_OPSGENIE_TAGS" | default ` {{ range .CommonLabels.SortedPairs }}{{ .Name }}:{{ .Value }},{{ end }}` {{ env "STAGE" | default "N/A" }} }}'

For such syntax ☝️ I get:

template: stringTemplate:306: unexpected "{" in operand

The tricky part is that I want to concatenate string {{ range .CommonLabels.SortedPairs }}{{ .Name }}:{{ .Value }},{{ end }} to another tag which is in environment variable. How to do that?

Mr.OOover 5 years ago

Hi,
I get the following error:

STDERR:                                                                                                              
  Error: no cached repo found. (try 'helm repo update'). error converting YAML to JSON: json: unsupported value: +Inf
  Error: plugin "diff" exited with error                                                                             
                                                                                                                     
COMBINED OUTPUT:                                                                                                     
  [debug] Created tunnel using local port: '44995'                                                                   
  [debug] SERVER: "127.0.0.1:44995"                                                                                  
  Error: no cached repo found. (try 'helm repo update'). error converting YAML to JSON: json: unsupported value: +Inf
  Error: plugin "diff" exited with error

if I do a "normal" helm diff upgrade it works with complaining.
helmfile template also work fine.
but I don't get helmfile diff to work

Has anyone stumbled into a similar problem?
I found this link, but it doesn't give that much information: https://github.com/helm/helm/issues/2909

joshmyersover 5 years ago

@Erik Osterman (Cloud Posse) (and others here) how are ya’ll doing Helmfile deployments via CI/CD, promoting per environment with per environment config? AFAIKR CP bundled a version of their helmfiles image with a per env container and slurped config values out of SSM with chamber, but doesn’t look to be the case anymore?

vgdubreaover 5 years ago(edited)

Hi folks, One more question it seems that I am facing issues using {{ .Environment.Values.<> }} in my releases: inside helmfile.yaml.
my structure is like this

vgdubreaover 5 years ago

to give more context we do Helmfile apply -f helmfile.yaml inside Helmfile.yaml we are calling Helmfile-common.yaml , this common file has all the environments and their values file defined.

vgdubreaover 5 years ago

Now I have some variable defined inside those environments that I need to use inside Helmfile but it seems {{ .Environment.Values.<> }} can't be used in Helmfile releases , any option anyone can suggest ?

muhahaover 5 years ago(edited)

More love for delete hooks support https://github.com/roboll/helmfile/issues/802#issuecomment-659938624 ? ( crd post delete )

Paul Catineanover 5 years ago

Is anyone here using terraform with helmfile ?

Paul Catineanover 5 years ago

I can't figure out how to send environment variables to existing helmfile files

Chad Ostlerover 5 years ago

helmfile sync --concurrency <x> seems to only run one at a time no matter what concurrency i set. is this a known issue?

Chad Ostlerover 5 years ago

i’m wondering if it’s due to the fact i have an umbrella helmfile of many nested helmfiles that we use labels to deploy selectively.

Andreas Lindhéover 5 years ago

Ohoy! I've just started using helmfile, and I'm a bit lost about secrets management.

Is it mandatory to have the helm-secrets plugin installed, even if my secrets are not encrypted? The secret I'm trying to include is not very secret, so I have no reason to add the complexity that comes with SOPS...

Oleksii Chuprynover 5 years ago(edited)

Hello! Is it possible to skip diff stage as part of apply command? Use case: override changes which were made manually (helm-diff doesn't show any diff in this case). So, i'm looking for the way of force execute helm upgrade --install . Thanks

Mathieu Frenetteover 5 years ago

Hey! 👋 I was wondering if there was a way to preview the template rendering of helmfile before actually applying? We often have templating issues that are hard to troubleshoot if we have no way to see what manifests were actually rendered by helmfile.

bradymover 5 years ago

helmfile template does that

Andreas Lindhéover 5 years ago

Can I include multiple values.yaml files in the values: section and they will be sequentially applied?

Andreas Lindhéover 5 years ago(edited)

Is the needs: directive broken? It complains to be that "default/y" depends on nonexistent release "x" while I definitely have a release x in my helmfile...

Dudi Cohenover 5 years ago

Did anyone figure out a way to run helmfile template only on specific charts/releases?

Michal Matyjekover 5 years ago

could not find a good answer: can I override name of the chart if syncing from helmfile.yaml?

something like helmfile -f charts/helmfile.yaml --name=notTheOneInHelmfile.yaml ?

Andreas Lindhéover 5 years ago(edited)

Is it possible to have helmfile label namespaces for me, or can it only put labels on Helm releases?

Graeme Gilliesover 5 years ago

How would one go about creating hierarchical environments in helmfile? Lots of use of yaml anchors?

Sam Buckinghamover 5 years ago

Has anyone else had issues with repositories not being updated? helmfile seems to be completely ignoring the repositories block in my yaml?

jason witkowskiover 5 years ago

Hey All, I'm having some really weird issues about context deadline exceeded from helmfile reaching my clusters. The issue does not happen if i manually query the cluster or use helm directly. Has anyone else ever seen this ?

jason witkowskiover 5 years ago(edited)

Another thing is we just went back and did a huge refactor across our repo to make massive use of templating since we deploy to many clusters across many regions and such. The performance of linting has degraded to point of not being able to run it in any meaningful way

June 2020 Browse by date August 2020