#kubernetes - July 2024

Sharing FYI....

https://learnk8s.io/exploring-the-kubernetes-instance-calculator

Hello everybody,

I had an issue few days ago during which an nginx deployment had a spike in 504 timeouts while trying to proxy the requests to the upstream php-fpm.

The issue lasted for 1h20min and resolved by itself. I was unable to find those requests reaching the upstream php-fpm pods.

I suspect that a node went away but the endpoints were not cleaned up. Unfortunately, I don't really have much evidence for it.

Anyone ever had similar issues where you had a large number of 504 between two services but you could not find any logs that would indicate those requests actually reached the other side?

Hey all. Thoughts on openshift for cluster management? Vs something like rancher. Are many folks running k8s on openshift? I can see the value-add, unsure of dollars at this stage but given majority of our footprint is the public cloud, using openshift instead of say AKS or EKS seems a little counter-intuitive

hey everyone, I wanted to know how you approach k8s upgrades. we are self hosting our K8s clusters (kubespray managed) in different DCs. I want to know how do you make sure the controller/operators in k8s does not have any breaking changes between say 1.x and 2.x? Do you surf through the changelog and look for keywords like breaking/dropped/removed? I want to know if there is some automated way or a tool to compare version change-logs and summarise the breaking changes. We already check for deprecated apis using kubedd, polaris and others, However this controller version change review is manual and error prone.

hi there, how are folks handling argocd deployments nowadays? im thinking about revamping our setup. right now we’re:

• hub spoke model using argocd app of apps pattern
• bootstrap the argocd namespace and helm deployment in terraform on the hub cluster
• point to our “app of apps” repository which uses helmfile
• let argocd manage itself
• janky CICD workflow script to add new child clusters under management
I think we want to continue letting app of apps manage our “addons” using helmfile, but wondering if there’s been any improvements in the initial bootstrap of argocd itself ( for the hub ) and the argocd cluster add portion for child clusters (perhaps via the argocd terraform provider?)

I would like to set a lifetime for pods. What are you using to achieve this?

Does anyone use cosign to sign their images and artifacts? How do yall deal with rotating those keys? Our concern is if a customer is using our public key to verify that the images running in their kubernetes clusters, if we rotate the signing keys, they will have to also...

anyone here want to help someone drowning in failure deploying some a helm chart for mage ai?

Does anyone know if there's an open sourced project that helps with creating, managing, verifying licenses for products we create?

still need help debugging why some of my pods are being killed (exit 137)