#kubernetes - October 2018

@Erik Osterman (Cloud Posse) yes sir!

https://blog.scottlowe.org/2018/09/28/setting-up-the-kubernetes-aws-cloud-provider/

(in addition to https://aws.amazon.com/blogs/opensource/provision-aws-services-kubernetes-aws-service-broker/, which was mentioned here before)

@Andriy Knysh (Cloud Posse) can you add some context?

this link https://aws.amazon.com/blogs/opensource/provision-aws-services-kubernetes-aws-service-broker/ describes an AWS service broker for k8s so we could provision some AWS resources from Kubernetes (e.g. Load Balancers or Route53 records)

this link https://blog.scottlowe.org/2018/09/28/setting-up-the-kubernetes-aws-cloud-provider/ shows some pitfalls when using it

Hello

[root@server kubernetes]# kubectl run hello-minikube --image=worpress The connection to the server localhost:8080 was refused - did you specify the right host or port?

How to fix this

hey @Gaurav - please share a little bit more about your env

are you on linux or mac?

@Gaurav the error usually means that kube config is not set or not found. Take a look here, hopefully some of the answers will work for you: https://github.com/kubernetes/kubernetes/issues/23726

Thanks @Erik Osterman (Cloud Posse) and @Andriy Knysh (Cloud Posse)

I am running this command in my centos7 desktop machine

Didn't got a fix yet :(

Ok, so you’ve already installed minikube? If so, how did you go about it?

@samh has joined the channel

Thanks @Erik Osterman (Cloud Posse)

@Erik Osterman (Cloud Posse) any idea about making https://www.telepresence.io work with geodesic

I don’t think geodesic stands in the way.

isn’t it just a glorified (but cool) reverse proxy?

i thought it replaces your pod for your service

and then proxies it back to your local workstation

that's correct, but it also modifies entries in iptables

oh really?

IMO (“in a perfect world”), seems like one should just replace the image with vendor/telepresence and pass some kind of KEY env

yes, I couldn't debug it completely but it worked flawlessly when running natively but within geodesic it doesn't able to resolve the proxied items

oh

are you saying the iptables stuff happens locally?

yes

aha, i see

yea, makes sense - now I understand what you want to do and how geodesic affects that

someway if geodesic can use the mac's iptable or sync with that

can you do something like you did with kubectl proxy?

port bind

actually there isn't a way to pass port binding

I can’t think of a practical way to keep it in geodesic

I think you’d need to export a kubecfg and run telepresence and kubectl natively

or run your apps in geodesic

e.g. nodejs app

i thought that also

will raise an issue about it on their git, let's see if they have a recommendation

anyone know of a neat secrets management thing that lets me back stuff up when using GKE + KMS

(I know etcdctl exists, I'm just wondering if that's literally how you do it)

The alternative is to encrypt all secrets in git using public KMS key. Then the system of record is git. Thus restoring is more about redeploying.

Anyone want to comment on https://medium.com/virtuslab/think-twice-before-using-helm-25fbb18bc822 ?

re: secure... Previously I used helm in a "Helm per namespace" model. Meaning, each app resided in it's own namespace, each namespace got it's own tiller, and each tiller got locked down by some very restrictive RBAC policies. Because of how Tiller operates, it effectively is a "giant sudo server", as the author puts it, but with the right policies, it can be used safely. What I most often hear is "what if someone gets on the tiller pod, they can do anything!" which is solved by the helm-per-namespace model. I wrote policies that allowed helm/tiller to create, update, and read within it's own namespace (meaning, i, as a human, was required to destroy something, because from day 2 on this wasn't required very often), and any attempt to retrieve any information from k8s outside of the namespace was denied.

the tiller is also slated to be deprecated in helm v3, for a strictly client-side operation

just like kubernetes, helm provides an interface for representing applications on kubernetes

kubernetes secrets, for example, are an interface for representing sensitive information. until kubernetes 1.8 these weren't encrypted and people scoffed at it too. these are fair critiques that don't go unnoticed. the thing is though, because these interfaces are defined well, the underlying implementations can be improved. that's what happened with kubernetes secrets.

the same will happen with helm and it won't require massive re-tooling.

(via one of the comments)

while I'm not crazy about Lua, it's worked out well for Nginx as the embedded language.

in terms of adding complexity, strongly disagree

I don't think gotemplating is easier to manage complexity. i think an actual language that could evolve into a DSL would be better. Lua has that potential.

also, v3 won't be limited to Lua. that's just the first alternative engine they are introducing. gotemplating will still be supported.

is this related to Flux?

not sure if i prefer terraform managing helm to helmfile managing helm

we’ve had to do a lot of conditional logic to support helm configuration which is supported by helmfile

but configurable values in helm is not easy. maybe with 0.12 it will be a viable alternative.

with terraform shared state as a service, it may win me over

a kops column would be nice

yea, I guess they are looking more at managed offerings

but kops would be good for comparison nonetheless

what’s going on with Alibaba Cloud ACK ? 🙂 (failed in many cases)

whats the go to deployment tool you guys are using

im looking for something that is simple for customers without any k8 experience

haha @Ryan Ryke, that would be Heroku 🙂

smarty pants

we use helmfile https://github.com/roboll/helmfile

and we have a collection of common helmfiles we use https://github.com/cloudposse/helmfiles

so you are running tiller and such on k8?

yes

also @Ryan Ryke, we use https://codefresh.io for CI/CD

in there, you can deploy a helm chart to a k8s cluster from the console

have you guys looked at jenkins x?

you can have a helm chart in any repositories (museums), they even offer their own Managed Helm repos

no, we did not use jenking x for that

its too bad the aws tools arent a little better suited

skaffold looks cool too

yea AWS is lacking a lot in k8s space

Looks like Deis and Dies Workflow are both no longer maintained

Those were the reason though helm exists

They are built to create a Heroku like experience for Kubernetes

Oh looks like Flynn is still alive and well https://github.com/flynn/flynn

Imo the were the underdog to deis. Deis got acquired by Microsoft and got abandoned shortly there after .

@Daren ☝️

scimmed it quick, is it free?

as far as I can tell

yea i went to the Kubecost website at the bottom of the article

and they had some pricing shennigans