gitops
Archived0494
Discuss continuous delivery of infrastructure
Archive: https://archive.sweetops.com/gitops/
E
erik12 months ago
archived the channel
Erik Osterman (Cloud Posse)about 1 year ago
I think we created too many watering holes
Erik Osterman (Cloud Posse)about 1 year ago
Haha, time for us to archive and consolidate conversations.
Zachary Loeberabout 1 year ago
GitOps no longer hip?
Zachary Loeberabout 1 year ago
So empty
joshmyersalmost 2 years ago
π How are folks doing gitops for Terraform, specifically promotion through environments? Previously terragrunt (because it was the closest thing to bare Terraform that could handle some orchestration) and the split repo approach. This was fine with Atlantis because split repo (live and modules) each env would be bumped to X.Y.Z release when ready. I'd like to do similar with a monorepo approach. git branches is a no go. Seen some folks automatically raising
env + 1 PRs, so like main goes to dev, that then automatically raises a PR for the same change to test etc...not terrible but can see some downsides...any others...?Adnanabout 2 years ago
Moving to enterprise GH, it was mentioned in the last office-hours, there might be permission issues causing workflows to fail.
Does anyone might know the docs link warning about this? Or maybe a blog post?
Does anyone might know the docs link warning about this? Or maybe a blog post?
Xu Pengfeiabout 2 years ago
KCL Biweekly Newsletter | KCL Biweekly Newsletter | v0.7.5 Released and Flux Integration for GitOps. Hi fellas! πΊ We have released KCL v0.7.5! For this version, we have written a blog to describe the updated content. Welcome to read and provide feedback! β€οΈ
Kfirabout 2 years ago
Hello, I'm seeking your advice on the best tools to integrate Terraform with GitOps. Options I'm considering include Atlantis, env0, and HashiCorp Cloud (theres more to consider?).
My primary criteria are ease of use, a comprehensive set of features, and reliability.
Budget constraints are not a concern in this scenario.
Which solution would you recommend based on these requirements?
My primary criteria are ease of use, a comprehensive set of features, and reliability.
Budget constraints are not a concern in this scenario.
Which solution would you recommend based on these requirements?
Isaacover 2 years ago
Weβre adopting GitOps with Argo CD and Iβm seeking insights on image promotion strategies. Currently, our pipeline builds and pushes images to ECR with a version tag on merging to main. A subsequent job updates deployments with these tags. For release candidates, we pull the image from the dev ECR, push it to the stage ECR, and update the stage deploymentβsame for production, but with a release tag.
Iβm considering simplifying this by having a single image build/push to a CI/CD ECR account, from which each environment would fetch the image upon tag creation. Is this unwise - help me see what I might not be considering?
Iβm considering simplifying this by having a single image build/push to a CI/CD ECR account, from which each environment would fetch the image upon tag creation. Is this unwise - help me see what I might not be considering?
Seanover 2 years ago(edited)
For GitOps of Helm, whatβs your preference between:
β’ π °οΈ CI tool renders to k8s manifests _(
β’ π ±οΈ CD tool (flux/argo/β¦) uses itβs own
And for both, whatβs your take on compatibility with Helm hooks?
β’ π °οΈ CI tool renders to k8s manifests _(
helm/helmfile template) -_> writes to git/s3/β¦ -> CD tool (flux/argo/β¦) pickups rendered manifests and applies them to cluster.β’ π ±οΈ CD tool (flux/argo/β¦) uses itβs own
helm functionality to render on the fly and apply to cluster.And for both, whatβs your take on compatibility with Helm hooks?
Tim Birkettover 2 years ago(edited)
GitOps... I'm still struggling to find any real reason to move from change-driven CI/CD with helmfile to something like ArgoCD or Flux.
With the CI/CD "push" approach, I can see deployment issues in a familiar interface, and a single pipeline view through to deployment. I can deploy through environments with visible e2e or smoke tests.
In the GitOps "pull" model I have yet another interface and bunch of logs to sift through. I have to workaround some things like CRDs.
IMO, most GitOps repos end up a bit of a mess over time π€
With the CI/CD "push" approach, I can see deployment issues in a familiar interface, and a single pipeline view through to deployment. I can deploy through environments with visible e2e or smoke tests.
In the GitOps "pull" model I have yet another interface and bunch of logs to sift through. I have to workaround some things like CRDs.
IMO, most GitOps repos end up a bit of a mess over time π€
Alexover 2 years ago
My company has dozens of application repos in with GitHub action pipelines push changes to other GitOps repos. I.e. Repo A1 pushes to repo A2, B1 > B2, C1 > C3, etc. What are the best practices for allowing application repos write access?
We are thinking of either
a) Use deploy keys. Each GitOps repo gets a unique key and then we add it as a secret to a corresponding app repo.
b) Create a GitHub application and install it into every GitOps repo. And then add its private key to all application repos as a secret. This creates a problem with the key being exposed to many repositories though. As a workaround, we can create a microservice to dynamically generate GitHub installation tokens and never expose the key itself.
We are thinking of either
a) Use deploy keys. Each GitOps repo gets a unique key and then we add it as a secret to a corresponding app repo.
b) Create a GitHub application and install it into every GitOps repo. And then add its private key to all application repos as a secret. This creates a problem with the key being exposed to many repositories though. As a workaround, we can create a microservice to dynamically generate GitHub installation tokens and never expose the key itself.
Xu Pengfeiover 2 years ago
I just published a blog on medium about KCL and GitOps https://medium.com/@xpf6677/implementing-gitops-using-github-argo-cd-and-kcl-to-simplify-devops-ad7940bef50 Welcome to read and provide feedback β€οΈ
Sudhish KRalmost 3 years ago
Hey Folks,
We at Dgraph Labs use Github as our VCS. And we have recently migrated our CI/CD setup to Github Actions. This was a huge win for us internally, especially in a startup setting like ours. Our wins were broadly in these 3 areas Compute Costs, Maintenance Efforts & Configuration Time.
With this new setup, we designed & developed Dynamic AutoScaling of Github Runners in house. We are thinking of open-sourcing this project. If there is any interest here - pls do reach out. We were able to save ~87% $$ of our Compute Costs with this setup.
Blog Link => https://www.sudhishkr.com/posts/20230217_dynamic-autoscaling-of-github-runners/
We at Dgraph Labs use Github as our VCS. And we have recently migrated our CI/CD setup to Github Actions. This was a huge win for us internally, especially in a startup setting like ours. Our wins were broadly in these 3 areas Compute Costs, Maintenance Efforts & Configuration Time.
With this new setup, we designed & developed Dynamic AutoScaling of Github Runners in house. We are thinking of open-sourcing this project. If there is any interest here - pls do reach out. We were able to save ~87% $$ of our Compute Costs with this setup.
Blog Link => https://www.sudhishkr.com/posts/20230217_dynamic-autoscaling-of-github-runners/
Maarten van der Hoefabout 3 years ago
Hi Everyone, GH Actions question..
Say I have two jobs with both the same matrix values, both multi-dimensional with the same matrix keys.
Now job 1 has output.
How can I refer to the output of job1 from job2, using the same matrix.values ?
Like
needs.job1[matrix.value][matrix.value2].output
Say I have two jobs with both the same matrix values, both multi-dimensional with the same matrix keys.
Now job 1 has output.
How can I refer to the output of job1 from job2, using the same matrix.values ?
Like
needs.job1[matrix.value][matrix.value2].output
yegorskiover 3 years ago
Has anyone migrated (not started anew) to gitlab.com? The source platform doesnβt matter but weβre moving from self-host GitLab. Looking to share experiences/swap war stories.
Eric Bergover 3 years ago
Seems like a reasonable place to ask how folks here manage slack configs, WRT managing tokens/keys/webhook URLs, etc., Doesn't seem to be a decent Slack TF provider.
DaniC (he/him)over 3 years ago
fyi added a bookmark here for "The path to GitOps" book, i think is a good foundation to have/ understand before diving into specific tools . Happy reading .
Matt McLaneover 3 years ago
I am looking for an article or youtube video that introduces the concept of gitops to a beginner. Anyone got any suggestions?
sheldonhover 4 years ago(edited)
@Mazin Ahmed Right now I'm just using lefthook and setting up a gitleaks hook to run on precommit and pre-push. It's not perfect. I've not centralized anything. You can use plugins in whatever CI tool you use to do more widescale work, but for now it's a way I've taken to get a small step towards quick code checks and pushing folks away from even dev certs and keys in the repos π
Mazin Ahmedover 4 years ago
Question: How do you use Git pre-hooks for identifying secrets on organization-level? I'm looking for ideas to detect sensitive commits before it's committed and pushed to Github. Any ideas how would you approach this on org level?
Issifabout 5 years ago
An article about our integration of ArgoCD and gitops workflow at work : https://medium.com/qonto-way/how-we-scaled-our-staging-deployments-with-argocd-a0deef486997
D
David Napierabout 5 years ago
Anyone know what these things are called?
Joan Portaabout 5 years ago
Hiya! do you have some comparison about using GitHubActions or GiLab pipeline as CICD?
Andrew Nazarovover 5 years ago
jafowover 5 years ago
anyone got a gitops flavored solution for triggering K8s Jobs that they like?
sheldonhover 5 years ago
Ok....
I want a cool win
Let's say I want to add a whitelisting entry to a security group in QA. Right now it's manual. Second option is terraform cloud cicd .... That said I am thinking of chatops or azure Pipelines or ssm automation runbook.
I want it to be super easy for team members to update their entry even if no GitHub knowledge so thinking maybe a messaging extension in teams to call an AWS lambda ? Easy to to do or learning curve steep. Also considered poshbot if I can do messaging extension but really wanted to use adaptive cards/form which makes sense to use lambda rest API for. Lastly I have mix of languages on the team and lambda would allow go, Powershell, and python contributions from team members vs poshbot probably making me run solo.
Messaging extension a good option or should I stick with GitHub runner and azure Pipelines concept, or gitops?
When I was talking through this with someone my preference would be pull request driven workflow. However I'm considering chat ops for this because a portion of those people don't even have experience with GitHub.
I want a cool win
Let's say I want to add a whitelisting entry to a security group in QA. Right now it's manual. Second option is terraform cloud cicd .... That said I am thinking of chatops or azure Pipelines or ssm automation runbook.
I want it to be super easy for team members to update their entry even if no GitHub knowledge so thinking maybe a messaging extension in teams to call an AWS lambda ? Easy to to do or learning curve steep. Also considered poshbot if I can do messaging extension but really wanted to use adaptive cards/form which makes sense to use lambda rest API for. Lastly I have mix of languages on the team and lambda would allow go, Powershell, and python contributions from team members vs poshbot probably making me run solo.
Messaging extension a good option or should I stick with GitHub runner and azure Pipelines concept, or gitops?
When I was talking through this with someone my preference would be pull request driven workflow. However I'm considering chat ops for this because a portion of those people don't even have experience with GitHub.
RBover 5 years ago
Working with multiple pull requests in
.github/PULL_REQUEST_TEMPLATE/ with 2 files general.md and kms_secrets.md . When I create a new PR, I expected to see a button to select which template like we see with issue templates. What could the issue be ?Milosbalmost 6 years ago
Is anyone using Spinnaker and store kubernetes manifests in Git (bitbucket) repository?
bradymalmost 6 years ago
I've done this a couple ways:
1. cron job that compares git branches to test envs, and shuts down any test envs for branches that have been deleted. This does, of course, rely on branches being deleted after they're merged. I know both github and gitlab now have an option you can set that will auto delete a branch after it is merged to master (or whatever you designate the "main" branch)
2. Gitlab CI has the concept of environments that are tied to branches. When a branch is deleted gitlab will run any jobs you've configured to run at the shutdown of an environment.
I've not used drone, but I'd be surprised if it doesn't have something like events or webhooks that you could use for this.
1. cron job that compares git branches to test envs, and shuts down any test envs for branches that have been deleted. This does, of course, rely on branches being deleted after they're merged. I know both github and gitlab now have an option you can set that will auto delete a branch after it is merged to master (or whatever you designate the "main" branch)
2. Gitlab CI has the concept of environments that are tied to branches. When a branch is deleted gitlab will run any jobs you've configured to run at the shutdown of an environment.
I've not used drone, but I'd be surprised if it doesn't have something like events or webhooks that you could use for this.
jedineeperalmost 6 years ago
Anyone done any work with feature branch based test environments? Seems easy enough to create them based on branch/pr but trying to find a convincing method to shut down those envs when the pr is closed / branch is merged.
Using drone for my ci platform but it doesn't seem to be tracking the right events for this - not sure many do?
Am i on the right track or is there a better logic for wrapping these up? :/
Using drone for my ci platform but it doesn't seem to be tracking the right events for this - not sure many do?
Am i on the right track or is there a better logic for wrapping these up? :/
Zachary Loeberalmost 6 years ago
gitlab running on-premise?
sheldonhalmost 6 years ago
Trying to find docs on this....
if I want to apply actions, templates, settings or anything across all my org in github, isn't there a
if I want to apply actions, templates, settings or anything across all my org in github, isn't there a
.github special repository you can create and all the child repositories inherit this, for instance with CODE_OWNERS for example? I thought there was a special repo for github that allowed this, like the .github name or something (not the folder in each github, I'm aware of that one)Erik Osterman (Cloud Posse)almost 6 years ago
Adding @U010XGY9B46 bot
Thomas Burtonalmost 6 years ago
Hello all
Have a scenario Iβd love some input with. We run around 18 microservices, a mixture of frontends and backends. As with many companies we have ended up with the monolithic microservice architecture and thus we often need to release things at the same time to prevent downtime on production. We have now successfully implemented FluxCD on our staging namespace. We have two more namespaces/environments, sandbox and production. We have not yet implemented flux in these namespaces. The issue we have to now is how best to implement flux and promote services from stage>sandbox>prod. It has been suggested we could use
Have a scenario Iβd love some input with. We run around 18 microservices, a mixture of frontends and backends. As with many companies we have ended up with the monolithic microservice architecture and thus we often need to release things at the same time to prevent downtime on production. We have now successfully implemented FluxCD on our staging namespace. We have two more namespaces/environments, sandbox and production. We have not yet implemented flux in these namespaces. The issue we have to now is how best to implement flux and promote services from stage>sandbox>prod. It has been suggested we could use
develop and master branches. If anyone could expand on that or add anything Iβd be super grateful. Hope you all have a great dayErik Osterman (Cloud Posse)almost 6 years ago
@UUB28NLDS help keep tabs! π
Erik Osterman (Cloud Posse)about 6 years ago
Haha fair enough
Nick Kampeabout 6 years ago
We've avoided helm for this long, not going to bring it in now π
Nick Kampeabout 6 years ago
potentially, if there's no better solutions
Nick Kampeabout 6 years ago
so I was going to create a mutating webhook service to attach a pod with them
Nick Kampeabout 6 years ago(edited)
Well I'm running k8s on the edge at a lot of sites that have specific configs that need to be applied
Erik Osterman (Cloud Posse)about 6 years ago
Helmfile is a swissarmy knife, so it would be possible to call curl
Erik Osterman (Cloud Posse)about 6 years ago
Aha! Thereβs your problem ;)
Nick Kampeabout 6 years ago
negative, Kustomize
Erik Osterman (Cloud Posse)about 6 years ago
Do you use Helmfile?
Nick Kampeabout 6 years ago
anyone have any clever ways to spin up a container that will fetch a remote config map and apply it?