#kubernetes - December 2020

Spot support in Managed Node Groups for EKS: https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-provisioning-and-managing-ec2-spot-instances-in-managed-node-groups/

^ I know this was discussed here a couple of times with people saying it was a blocker

https://get-kbld.io/ -> this and all the carvel tooling may be interesting to keep an eye on.

Stupid question here... When an image is pulled by the Kubelet, is this done with the default service account or is it done with the whatever service account is specified on the pod (default when there is nothing specified on the pod)? I'm wondering if all service accounts need image pull secrets setting or just the default service accounts 🤔

Does anyone else feel that kube-system gets overused sometimes? What are people's strategies for installing system related tools like cluster-autoscaler, kube-downscaler and other operators / controllers? Single namespace? Namespace per controller? Something else?

I am working on the upgrade implementation for a legacy application we host in kubernetes. Part of the upgrade procedure is going to require manipulation of k8s resources (configmaps, potentially ingress resources) at strategic points during the upgrade lifecycle. I am planning on using helm hooks running Jobs to do this; my question/concern is: is bad practice to have a pod manipulating k8s resources? If it's not, what is the best way to accomplish it - just have kubectl available within the pod?

We currently build our product on AWS and we're looking to also support GCP. We use ECS backed by EC2, and using GCP means moving to K8s. I know it'll be a lot of work (a lot) so we first want to get a 10,000-ft view by mapping all our current AWS concepts to their GCP/K8s equivalents.

Anybody have pointers to useful guides on this conversion?

I’m setting up Istio over eks. Wanted to ask what’s the best strategy to have an encrypted tls connection between a client outside the cluster and a pod (ingress). I’m managing certificates on AWS ACM and it seems that elb has support for that using annotations, but according to my understanding that’ll lead to an unencrypted traffic between the elb and istio gateway. Any opinion about that would be extremely helpful.

I am tightening up permissions on my EKS cluster (1.17) for my devs to manage k8, both in a read-only as well as more of an admin role, but I'm having difficulty finding the right policies to allow k8 mgmt. Can anybody point me in the right direction to help me write the policies I need for my users to talk to k8s? Thanks!

We have some AWS Lambda functions that I'd like to migrate to run in our k8s clusters (EKS). Has anyone done this and can offer toolchain recommendations? There seem to be a lot of options: OpenFaaS, Fission, Kubeless, ...

anyone have any kubernetes feature gates that you’ve turned on that you love and we all should know about? 😃

I am deploying two apps, server + client. The client needs to configure a URL that points to server, is there a way I can use reference the svc?
The server has, it's ports can be reached from $RELEASE-dgraph-alpha-$IDX.$RELEASE-dgraph-alpha-headless.$NAMESPACE.svc

I'm not sure how to get this to work:

{{- if .Values.script.enabled -}}
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "dgraph-lambda.fullname" . }}-config
  labels:
    {{- include "dgraph-lambda.labels" . | nindent 4 }}
data:
  script.js: {{ .Values.script.script }}
{{- end -}}

This get me:

[ERROR] templates/config.yaml: unable to parse YAML: error converting YAML to JSON: yaml: line 13: mapping values are not allowed in this context

Hey EKS folks — I’m finding a pretty consistent worker node downtime pattern: I have a worker node group of 4 and after an undefined number of days, the oldest worker node will go into an Unknown state. The node will go into a <http://node.kubernetes.io/unreachable|node.kubernetes.io/unreachable>: NoSchedule + <http://node.kubernetes.io/unreachable|node.kubernetes.io/unreachable>: NoExecute state, the Kubelet stops posting node status updates to EKS, and I can no longer seem to be able to access that particular node.

Has anyone seen this pattern? I’m just starting to look into it and figured it’d be quick to post about it here before I jump all the way down the rabbit hole.

Hi anyone have any step by step guides with easy to follow contents to setup a kube cluster?

Hey everyone, what do you guys use to secure access to internal resources (kube-dashboard, grafana, argo, etc). Just port-forward? VPN? I'm looking at exploring Pomerium, but just wondering how other people do it

Does anyone use Google Skaffold, https://tilt.dev/ or another developer iteration/productivity tool? I'm currently stumbling around with evaluating Tilt.

I know I know, I should be with family and such but I'm injured in bed with nothing to do so I'm playing with EKS, so the questions is to node_group or not to node_group?(I'm new to this and I want to play with istio after I have the cluster running)

for those using Service meshes any pros and cons between istio and Gloo ( and maybe others?)?

for deploying apps in K8s using helm charts what are those recommended tools needed to create ( link, test ) etc you guys use? I'm new to this and I want to know what should I use to go from repo to infra to deploy ( CRDs and such) gitOps all the way basically

Anyone here knows about Kubernetes deployments with helm in air-gapped systems?

What will be the recommended way when using EKS cluster for lets say for CD/CD or Control plane management and yo wanted to keep the ingress in a private subnet, will that work? ( we keep our CI/CD systems behind vpn and since I was playing with ArgoCD I was using the port-forwarding option)

What would be a right approach most effective open-source for running Kubernetes on KVM, Hypervisor, LXC in house on a home lab?

By any chance, does anyone here have a multi-region kubernetes setup that still uses wildcard DNS? I have a single cluster with hundreds of ingresses like foo.example.com or bar.example.com and I had been thinking about moving to a multi-region setup where half of the ingresses would live in us-east and half in us-west, but would like to keep the wildcard dns setup as to not need to create a bunch of route53 records. I can’t use Route53 geo-based routing as users that have their site hosted in us-east could be accessing their site from a different location (i.e. california). To clarify, the reason that I want to add a cluster in a second region is to minimize blast radius and not for redundancy (foo.example.com would only live on the us-east cluster OR the us-west cluster but not both)