#kubernetes - March 2019 | Slack Archive

An automated drain/cordon strategy of rolling updates for EKS is lacking. As such, we have preferred working with kops. Until such a strategy exists, EKS+terraform we prefer to stick with kops

Erik Osterman (Cloud Posse)almost 7 years ago

Even eksctl does not support rolling updates yet

Dave Yualmost 7 years ago

running into this error with external dns and unlimited staging environments. are there any strategies for dealing with this besides being more restrictive with naming? in this case I’m using the git branch name.

"InvalidChangeBatch: [FATAL problem: DomainLabelTooLong (Domain label is too long) encountered with '14bd77e7-db57-4000-8250-d89cd97cef51-staging-db-myapplication-backend']\n\tstatus code: 400, request id: c7a58d7e-3fa5-11e9-9ea2-799474e23419"

Erik Osterman (Cloud Posse)almost 7 years ago

Aha, yea need normalization

Erik Osterman (Cloud Posse)almost 7 years ago

What works well is to use slashes as a delimiter in your branch names

Erik Osterman (Cloud Posse)almost 7 years ago

E.g. epic123/my-fix

Erik Osterman (Cloud Posse)almost 7 years ago

Use the first field for the service disambiguation

Erik Osterman (Cloud Posse)almost 7 years ago

This also allows you to have multiple PRs deploy to the same namespace

caseyalmost 7 years ago

has anyone every tried out openshift origin? Any opinions on it?

Erik Weberalmost 7 years ago

helped a client once, i wouldn't do it again but i don't know how it compares to other managed k8s solutions

Erik Weberalmost 7 years ago

first impression is that it's hard to get the initial configuration right if you're not familiar with it, think we ran the installation playbook like 10 times before it worked successfully

Erik Weberalmost 7 years ago

the actual installation is handled by ansible and pretty straight forward

Erik Weberalmost 7 years ago

when shit hits the fan you have 3 layers to troubleshoot - plain docker, kubernetes and openshift

Ramesh Kumaralmost 7 years ago

is there any way to print all the kubernetes resource objects?

johncblandiialmost 7 years ago

So, anyone running Jenkins get docker images to build without using docker.build commands from the plugin?

We need to run TF commands to get state, pass in build args, etc and our Makefile handles this. I’d rather not recreate that in a Jenkinsfile. Running 1 simple: make release ... is much easier.

Thoughts?

btaialmost 7 years ago

does anyone here have a pattern to preload docker images into their cluster before deploys

btaialmost 7 years ago

scenario where you have a weekly cadence (i.e. release every tuesday) and you could preload all the worker nodes with the images the monday night before?

Erik Osterman (Cloud Posse)almost 7 years ago

How large is the cluster? Is this something exasperated by the size of your cluster when pulling the images?

btaialmost 7 years ago

@Erik Osterman (Cloud Posse) we can assume ~50 worker nodes

i5okiealmost 7 years ago

anyone know anyone for an invite to slack.k8s.io ?

Erik Osterman (Cloud Posse)almost 7 years ago

You have filled out the form?

btaialmost 7 years ago

they had an issue where bots were spamming channels with NSFW stuff so they had to shut down slack registration

i5okiealmost 7 years ago

I don’t know any contributors or anyone to mention as sponsor

Maarten van der Hoefalmost 7 years ago

Hi, is there anyone who used https://www.telepresence.io/ ? I’d like to gather experiences and/or possible alternatives.

Mark Grubealmost 7 years ago

I haven’t used telepresence, but I was also researching tools for local dev. This one looks really interesting: https://github.com/windmilleng/tilt

Ryan Smithalmost 7 years ago(edited)

Any one uses helm plugins? Curious of your experiences with them. Came across this and it was interesting: https://github.com/totango/helm-ssm

Erik Osterman (Cloud Posse)almost 7 years ago

Love the helm-git and helm-s3 plug-ins

Erik Osterman (Cloud Posse)almost 7 years ago

Also helm-diff

Ryan Smithalmost 7 years ago

👍️

endofcakealmost 7 years ago

Anyone using makisu or kaniko for Docker builds?

nutellinoitalmost 7 years ago

Hello everyone, it is safe to update worker nodes to the latest minor kubernetes ami version? I got EKS master plane on version 1.11.8 and worker nodes on 1.11.5 and 5 autoscaling group attached to the cluster

nutellinoitalmost 7 years ago

nevermind, i updated ami id, the minor version is still the same, 1.11.5

btaialmost 7 years ago

any one here familair with alb ingress controlers?

Erik Osterman (Cloud Posse)almost 7 years ago

@tamsky @Igor Rodionov ☝️

tamskyalmost 7 years ago

@tamsky has joined the channel

Erik Osterman (Cloud Posse)almost 7 years ago

https://itnext.io/kubernetes-authorization-via-open-policy-agent-a9455d9d5ceb

Abhsihek Jaisinghalmost 7 years ago

Hey there guys! Are you still using helmfile? Had some questions regarding the same

Abhsihek Jaisinghalmost 7 years ago

Sadly, couldn't find any official community / channel for helmfile elsewhere 😞

mmuehlbergeralmost 7 years ago

Here’s one in this Slack: #helmfile

Erik Osterman (Cloud Posse)almost 7 years ago

Welcome @Abhsihek Jaisingh ! You’ve come to the right place as @mmuehlberger points out. Looks like @mumoshu is going to update the README for Helmfile so it will be easier for others to find.

Igor Rodionovalmost 7 years ago

https://github.com/aquasecurity/kubectl-who-can

Igor Rodionovalmost 7 years ago

Check this up @Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse)almost 7 years ago

That’s awesome!

Erik Osterman (Cloud Posse)almost 7 years ago

@Jeremy G (Cloud Posse) ☝️

btaialmost 7 years ago

can we have an internet facing ALB that routes traffic to k8s worker nodes in private subnets?

Erik Osterman (Cloud Posse)almost 7 years ago

Yes

Erik Osterman (Cloud Posse)almost 7 years ago

That's what we are doing

Erik Osterman (Cloud Posse)almost 7 years ago

https://github.com/cloudposse/helmfiles/blob/master/releases/aws-alb-ingress-controller.yaml

btaialmost 7 years ago

man i am having lots of trouble getting alb ingress controller to work on my cluster

btaialmost 7 years ago

i know you guys are using kops though (not eks)

btaialmost 7 years ago

and i just read this: https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html

btaialmost 7 years ago

Note

Internet-facing load balancers require a public subnet in your cluster. Worker nodes also require outbound internet access to the Amazon EKS APIs for cluster introspection and node registration at launch time.

Tim Malonealmost 7 years ago

how did you set up your VPC? with the cloudformation template referenced in the EKS getting started docs?

Tim Malonealmost 7 years ago(edited)

(although i think that only creates public subnets… so might need some additional set up for private subnets for workers, but ALB ingress should still work once you’ve done that…)

btaialmost 7 years ago

with terraform

Tim Malonealmost 7 years ago

and it’s got the relevant config mentioned in the eks getting started docs? (subnet tags and such)

btaialmost 7 years ago

yeah it does

btaialmost 7 years ago(edited)

i wrote a blurb in #sig-network I can copy/paste here

Erik Osterman (Cloud Posse)almost 7 years ago

@Daren @Ram https://github.com/cloudposse/helmfiles/pull/107

Erik Osterman (Cloud Posse)almost 7 years ago

@Max Moon @Dave Yu ☝️

Erik Osterman (Cloud Posse)almost 7 years ago

shout out to @Jeremy G (Cloud Posse) for whipping this up

deftunixalmost 7 years ago

Hi all, is anyone of you using https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/2.3.0 to provision EKS cluster? I am experiencing an issue in the service type LoadBalancer. the cluster is not creating the balancer

Andriy Knysh (Cloud Posse)almost 7 years ago

Let’s move to #terraform-aws-modules . Ping @antonbabenko , he is the creator of the module

Erik Osterman (Cloud Posse)almost 7 years ago

https://zwischenzugs.com/2019/03/25/aws-vs-k8s-is-the-new-windows-vs-linux/

Erik Osterman (Cloud Posse)almost 7 years ago

https://learnk8s.io/blog/kubectl-productivity/?utm_source=share&utm_medium=ios_app

Marcel Pivaalmost 7 years ago

hello guys

Marcel Pivaalmost 7 years ago

somebody already have a problem with metrics using eks ?

Marcel Pivaalmost 7 years ago

the problem is in the hpa addon. from kubernetes-incubator

Marcel Pivaalmost 7 years ago

https://github.com/DirectXMan12/k8s-prometheus-adapter/issues/119

caseyalmost 7 years ago

anyone have suggestions on a good rate-limiting gateway for kubernetes? I have been looking at kong, ambassador, and express-gateway, not sure which one to go with or if there is a better choice

Erik Osterman (Cloud Posse)almost 7 years ago

@casey istio provides rate limiting (envoy)

Erik Osterman (Cloud Posse)almost 7 years ago

https://istio.io/docs/tasks/policy-enforcement/rate-limiting/

Erik Osterman (Cloud Posse)almost 7 years ago

plus with istio you get so much more than just that

jtbrayalmost 7 years ago

@Erik Osterman (Cloud Posse) I was watching one of Kelsey HT's keynote speeches, when he whips out his phone & starts using Google Assistant to manage some network services (via Istio)!! I'd love to see a well formed write-up discussing similar implementations &/or pitfalls/concerns.

jtbrayalmost 7 years ago

(has me very excited to control everything by voice)!!

jtbrayalmost 7 years ago

here it is... https://www.youtube.com/watch?v=6BYq6hNhceI

February 2019 Browse by date April 2019