#kops - June 2019 | Slack Archive

kopsArchived

47 messages

Discussions related to kops for kubernetes
Archive: https://archive.sweetops.com/kops/

Feb 2019

June 2019

Jul 2019

Janalmost 7 years ago

is any one using the <http://dns.alpha.kubernetes.io/internal|dns.alpha.kubernetes.io/internal> annotation on nodes with success?

Erik Osterman (Cloud Posse)almost 7 years ago

Not sure about that annotation

Erik Osterman (Cloud Posse)almost 7 years ago

What does it do?

Janalmost 7 years ago

Adds a dns record with the internal (vpc ip) of the nodes

Janalmost 7 years ago

No way to filter nodes though

Janalmost 7 years ago

Not the solution im after, have a few other ideas

Erik Osterman (Cloud Posse)almost 7 years ago

What is the solution you are after?

Janalmost 7 years ago

A maintained a record listing the vpc ips of all instance in a set instance group

Janalmost 7 years ago

Cassandra dedicated nodes

Janalmost 7 years ago

Exploring a similar solution using external-dns and host port

Janalmost 7 years ago

Tim and I are going to start pushing PR's up to you

Janalmost 7 years ago

I think time might already have one in for fixing your race condition with multi vpc peering

Erik Osterman (Cloud Posse)almost 7 years ago

wow, that would be cool!

Darenalmost 7 years ago

We are looking at upgrading from Kops 1.11 to 1.12. The upgrade instructions mention that it is a disruptive upgrade without going into details of how much. Is there anyone who has gone through it and can share their experience?
cc @Jeremy G (Cloud Posse)

Erik Osterman (Cloud Posse)almost 7 years ago

@btai @Rohit Verma @Jan ☝️

btaialmost 7 years ago

@btai has joined the channel

Rohit Vermaalmost 7 years ago

@Rohit Verma has joined the channel

btaialmost 7 years ago

unfortunately still on 1.11.9 for our kops clusters

btaialmost 7 years ago

also im not sure if i will run into this issue even when i do upgrade to 1.12 as my clusters are ephemeral (i would spin up a new 1.12 cluster and deploy/cutover to it)

btaialmost 7 years ago

Technically there is no usable upgrade path from etcd2 to etcd3 that supports HA scenarios, but kops has enabled it using etcd-manager. Nonetheless, this remains a higher-risk upgrade than most other kubernetes upgrades - you are strongly recommended to plan accordingly: back up critical data, schedule the upgrade during a maintenance window, think about how you could recover onto a new cluster, try it on non-production clusters first.

btaialmost 7 years ago

it almost sounds to me that spinning up a new cluster is prob the safest way forward, but im trying to imagine the way you guys are terraforming the cluster/env might make it hard to do that type of blue/green cutover?

Darenalmost 7 years ago

blue/green is difficult for us right now. We are already on etcd 3, but no TLS or etcd-manager. We also use calico 😞

Erik Osterman (Cloud Posse)almost 7 years ago

Can you use route53 to route traffic to both cluster?

Erik Osterman (Cloud Posse)almost 7 years ago

that would give you a fall back plan

Erik Osterman (Cloud Posse)almost 7 years ago

or use external CDN (e.g. cloudflare) with multiple origins

Darenalmost 7 years ago(edited)

We still use VPC peering to bridge kops and our backend vpc

Erik Osterman (Cloud Posse)almost 7 years ago

oh, but you can peer the VPC to both k8s clusters

Erik Osterman (Cloud Posse)almost 7 years ago

so you create a new kops vpc

Darenalmost 7 years ago

Yes, I said "difficult" not impossible 🙂

Erik Osterman (Cloud Posse)almost 7 years ago

haha, true

Erik Osterman (Cloud Posse)almost 7 years ago

though this could be a good capability to support

Erik Osterman (Cloud Posse)almost 7 years ago

even for future upgrades

Darenalmost 7 years ago

yes

Erik Osterman (Cloud Posse)almost 7 years ago

at the rate k8s moves, this won't be the last breaking change

btaialmost 7 years ago

thats exactly what we do

btaialmost 7 years ago

kops cluster in its own vpc, peered to our database vpc, new cluster comes up will also vpc peer into db.

btaialmost 7 years ago

allows for quick route53 cutover/ you can also do a weighted cutover via route 53 and you have a pretty fast rollback strategy (point route53 back to old cluster)

Jeremy G (Cloud Posse)almost 7 years ago

Discussion of some of the options for upgrading etcd (none great): https://gravitational.com/blog/kubernetes-and-offline-etcd-upgrades/

Jeremy G (Cloud Posse)almost 7 years ago

Step-by-step instructions for upgrading kops cluster by replacing it. Probably best for 1.11 to 1.12 upgrade. (I've never tried it. I have not had to upgrade a cluster from 1.11 to 1.12 yet.) https://www.bluematador.com/blog/upgrading-your-aws-kubernetes-cluster-by-replacing-it

Danalmost 7 years ago

Are people using kops for GCP or Azure? or are people using Kubespray for more multi platform

btaialmost 7 years ago

last i checked (late 2018) kops didnt support azure, but AKS has been plenty good.

Erik Osterman (Cloud Posse)almost 7 years ago

I wouldn't use kops even for GCP

btaialmost 7 years ago

however azure postgres (and azure as a whole) have not had great uptime imo

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

I feel like the best option is to use the best tool for the platform. using any kind of generalized tool will likely not give you all the extra jazz provided by the platform.

btaialmost 7 years ago

AKS has been chugging along though but if you have other dependencies within azure 😅

Erik Osterman (Cloud Posse)almost 7 years ago

e.g. on google, I'd prefer to operate GKE over GCP+Kubernetes

btaialmost 7 years ago(edited)

same, AKS is great cause it feels to me fully managed (as opposed to EKS which is highly configurable and even the generic case takes more effort to spin up)

February 2019 Browse by date July 2019