archived the channel

Very cool feature here, super easy self hosted runners for GitHub actions, using Codebuild...
https://aws.amazon.com/about-aws/whats-new/2024/04/aws-codebuild-managed-github-action-runners/

why would cloudposse/github-action-secret-outputs@0.1.0 throw this?

gpg: decrypt_message failed: Unknown system error

Here's one, for you release-engineering types.
What's the approach that most folks take these days for handling db migrations. Laravel, or whatever. Do folks tightly couple the migrations with the application, applying them at the time of app code deploy? I still see this frequently, and it seems aweful.
My answer is the same as always -- to manage the DB schema/queries/etc., as their own versioned products. This way if you know your app is going to need a new column you can make that change before the app is deployed. As you would do with any dependency. LMK, I'm interested in finding out what others see/do.

@Erik Osterman (Cloud Posse) do you have any idea?

Hello please is their a way i can automatically generate a new token for my self hosted runner before it expires?

Hello, how do you deal with Docker in Docker builds that are being shared by multiple repos? I've got GitlabCI repos that share one Gitlab runner. Both repos have multiple stages and last stage is pruning volumes and images. If both pipelines run at the same time one of them finishes and prunes volumes that are being used by other pipeline at the same time.
Should I not prune volumes?
I know I can delete images older than x, but volume command does not have the same flag

This message was deleted.

For updating to newer AMIs (custom or public), what’s your release strategy to release up through dev->int/test->stage->prod?

We want to get to the point of fully automatic releases (with delays, and maybe PR approvals up to each environment).

Hey Folks,

We at Dgraph Labs use Github as our VCS. And we have recently migrated our CI/CD setup to Github Actions. This was a huge win for us internally, especially in a startup setting like ours. Our wins were broadly in these 3 areas Compute Costs, Maintenance Efforts & Configuration Time.

With this new setup, we designed & developed Dynamic AutoScaling of Github Runners in house. We are thinking of open-sourcing this project. If there is any interest here - pls do reach out. We were able to save ~87% $$ of our Compute Costs with this setup.

Blog Link => https://www.sudhishkr.com/posts/20230217_dynamic-autoscaling-of-github-runners/

Hi all, had an interview today and got an interesting question that i would like to hear what are you think about it … theres a 100 repositories, 99 of them are microservices , 1 one of them is a nuget package repository (all of the services are dependent on this package) if someone changes the first repo, how would you deploy all of the rest of the microservices? For me it was unreasonable question since i thought to myself .. each developer should update the depndency version of his microservice… just want to hear your opinion on this

Currently using aws codebuild/pipelines/ECR and trying to improve the build times thru faster provisioning, caching, faster disk etc. Codebuild doesn’t seem to support FSX, and can’t run on an nvme server that I can determine. Anyone have any recommendations for an alternative build setup, thinking maybe self-hosted circleci or github actions? I can’t find any good performance comparison, but ideally high performance to save some minutes on some of these node/rails builds.
We have aws provisioning times taking 30-40s over and over sometimes when we’re using the same latest image over and over, and that seems silly.

Anyone have any experience with https://www.dependencies.io/ especially vs Dependabot?

fyi, issues with github actions at the moment... https://www.githubstatus.com/

Hello everyone, I am building a CI/CD pipeline for a product and I am confused with a few things.

I have so far worked in a system where I used to do "code promotion" for environment progression ie each branch pointed towards a certain env and PRs between the branches but very recently I read about "artifact promotion" and I feel like it is sensible thing to do and want to give it a try.

Now for my microservices, I am able to manage it by keeping Docker image for each env, free from any env specific variables and supplying env variables to my pod directly. It all works. But for my frontend, I am hosting it using S3 & CloudFront and I am using Next JS f/w and the way env variables work in Next JS is that we need to supply them at build time and they get embedded in the export/dist.

How do I do "artifact promotion" in such cases, specially when the env variables are different for each environment.

PS: I know this question is very specific to my use case. Apologies if I am asking it at a wrong place!

Any recommendations on tools to build and release mobile apps? I was recommended https://fastlane.tools/ but wanted to get other inputs as well.

For context, we are building our mobile app using React Native

Does anyone have good advice on best practices invoking integration tests in pipe with external user supplied ephemeral infrastructure dependencies?

We recently moved to a monorepo in gitlab. I setup the pipeline to run jobs based on what files have changed using rules:changes. Unfortunately I missed this gem in the troubleshooting section until after the changes were released:

The changes rule always evaluates to true when pushing a new branch or a new tag to GitLab.

So now every new branch is building and deploying all apps instead of just the one(s) with changes. Anyone run into this? Any recommendations on fixes?

I'm hoping to avoid adding some sort of check to all jobs as to whether or not they should run and exit early if not.

New tool for managing self hosted GitHub runners, https://cloudbase.it/manage-your-own-github-runners-using-garm/

My team hates dislikes the /test all chatops that I set up in order to run GitHub Actions that require secrets from forks. (I copy/pasted CloudPosse's slash-command-dispatch.yaml) We looked at enabling that setting that requires approvals before pipeline runs from forks, but it only requires an approval once from a given user. It's a mitigation to prevent annoyance and frivolous pipeline runs, not to protect the repo's secrets.

Is there any other option? I'm sure you guys have looked at this a lot.

Oidc + GitHub actions is amazing, this looks like a great enhancement... https://twitter.com/__steele/status/1487991518493556739?t=BRZHQN3kigLcs6euv5QHwA&s=19

This is likely not the right place to post this questions, but is anyone using something besides Artifactory to host / store their Artifacts? If so, what are you using?

hi folks, in one of the office hours or other videos it was mentioned a CD pattern using GHA instead of codefresh/ spacelift. If i remember correctly it was said that a pattern of below could be achieved:

CI:

• TF config or modules in one git repo
• GHA to build and upload artefacts - ie lambda docker/ layer/ zip file to S3/ ECR (not use tf lambda modul to deal with packaging
CD:

• a different / standalone git repo which will run TF plan/ apply to v envs.
Q:

What i don't get how the source repos (in case of poly-repo where various TF modules are scattered in N GH org repos) will:

• be synced to CD git repo? What sort of dir structure layout will be on the CD repo to cover for multiple deployments for different root modules ?
• any metadata consumed by CD and generated by CI - how will that be available ? i.e each lambda zip file or docker image / or TF code will generate a new tag/ release ... that needs to be consumed somehow by CD GHA workflows no ?
anyway i forgot who made the claim about using GHA as CD, will love to understand more the pattern

hi folks, has anyone managed to get an idea the cost between GHA/ CodeFresh & Spacelift? Asking as i've been using GHA which is dead cheap however when i'm looking at the others i see 250$/m and that limited to 5 users.

anyone run into issues with github-actions and using an env value that starts with https://? e.g.

    env:
      REPO_ENDPOINT: 'https://...'

it keeps stripping the https:// from the env in the log, but i can't find any docs on that behavior. and i'm pretty sure the yaml is fine?

ugh. testing this out and the ‘reusable workflow’ must be in a public repository

Is anyone aware of any self-hosted or paid solutions for github workflow dashboards across an entire organization of repositories? I’ve previously found meercode.io and they seem alright, although they don’t seem to be doing much in the way of updates on their proposed feature list (particularly the custom domain names)

Hey there! I added a sample on how to use github-status-updater with argo-workflows: https://github.com/cloudposse/github-status-updater/pull/18/files

Hi folks, we have our application on Azure DevOps and have to deploy AWS infra using terraform again with Azure Pipelines, do you have any recommendation around this or examples that I can use? Thanks

looks like a nice enhancement... https://github.blog/changelog/2021-08-25-github-actions-reduce-duplication-with-action-composition/

Hi
Any bitbucket cloud users ? How do u manage AWS Multi Account Pipeline deployments ?

I want to avoid adding AWS Keys per repo but we only have 1 workspace in Bitbucket Cloud and I don’t think it will be secure to add AWS Keys which can assume role into other AWS Accounts accessible to all the projects and repo under the workspace umbrella .

Hello , is there any way to use one cloudwatch rule for all the repos in code-commit.
Thanks in advance!!

Would that help ?

If that’s the case, should I ask at a bare minimum access to cloudpossee/contributors team ?

Is it because I am not in the cloudposse organization in any of the teams ? Does it only react to org members comments ?

Curious why /test all GitHub comments don’t trigger bot here: https://github.com/cloudposse/terraform-aws-sns-topic/pull/35#issuecomment-881782306

I know for sure I’ve seen a good tool for this before:

Tracking checksums (or content) of remote URLs/files? (Most cases it’s probably git repos). For example to refuse build until you’ve accepted the new checksum of upstream or so… thoughts?

Just want to get some feedback from everyone.
1. we have a python backend and a react front. currently everything is in aws. my architect recommended this setup. What do you think? Is this an easy task to do in aws?
2. Does anyone have experience with Trend Micro Cloud Conformity. Do you recommend it or not?
3. For SCA and SAST tools, what would be good libraries or tools for python and react code base?

[thread] 🧵 Branching & PR Methodology for Improving Velocity and Reducing Size of PR

I normally use main with short lived feature branches having used this successfully in a devops oriented role..... see thread for details. Could use some ideas to setup remote team for better success.....

kinda relevant to this channel, what would you expect this Makefile to output:

BAR ?= bar

bar: RECURSIVE = recursive
bar: SIMPLE := simple
bar: CONDITIONAL ?= conditional
bar: ;

foo: FOO ?= foo
foo:
	@echo [make vars]: FOO = $(FOO), BAR = $(BAR), RECURSIVE = $(RECURSIVE), SIMPLE = $(SIMPLE), CONDITIONAL = $(CONDITIONAL)
	@echo [env vars]: FOO = $$FOO, BAR = $$BAR, RECURSIVE = $$RECURSIVE, SIMPLE = $$SIMPLE, CONDITIONAL = $$CONDITIONAL

when running FOO=fee BAR=fi CONDITIONAL=fo make foo?

anybody else facing issues with Github actions today?

I am getting some errors and my jobs won't start:

GitHub Actions has encountered an internal error when running your job.

is anyone here using meercode.io as a CICD dashboard for stuff like github actions?

Not sure if it’s the best option but here we go.
Codefresh is triggered on every commit and can have single or multiple files changed on the branch.
I’d like to trigger step for every file changed. Kinda iterate over a list and run step for every iteration.
For now I’m forced to iterate list of changed files in single step which is reducing visibility of pipeline.

It would be most convenient to stick with pure codefresh solution instead of adding labels to PR or whatno but I’m all ears to different solutions.

@Maxim Mironenko (Cloud Posse) has joined the channel

Oh this looks really useful... https://blog.gruntwork.io/introducing-git-xargs-an-open-source-tool-to-update-multiple-github-repos-753f9f3675ec

Does anybody know any tool allowing you to have a big picture of services installed to the cluster/namespaces and their versions (whether taken from labels or docker tags). Just that dead simple. I bet I came across one some time ago, but it slipped my mind.

Or, generalising: what are better approaches to have a big picture (graphical or table representation) of services running, say, in a Kubernetes cluster which can be consumed by non-technical people

Hello,
how looks a CICD pipeline for Ansible, which check would you run before each deployment ?
do you run preflight check , or ansible-playbook with --check option ? output result as junit reports ?

Could someone point me at a repo where you’re using vendir to pull in parts of a Github Actions workflow? I saw this on the cloudposee reference docs and we’re curious to see what this looks like:

GitHub Actions do not currently support importing remote workflows. Using vendir we can easily import remote workflows.