#gitops - October 2018 | Slack Archive

gitopsArchived

79 messages

Discuss continuous delivery of infrastructure
Archive: https://archive.sweetops.com/gitops/

Sep 2018

October 2018

Jan 2019

jarvover 7 years ago

@jarv has joined the channel

mallenover 7 years ago

@mallen has joined the channel

Maarten van der Hoefover 7 years ago

Hi everyone, I wonder how most of you are doing secrets management. In a stack I'm working on the applications their env vars (ssm) are populated through terraform. Secrets like API keys are also stored in git but have been encrypted by KMS. Those secrets will be decrypted by terraform using aws_kms_secrets. My belief is that this is secure as envelope encryption would make the encrypted hash useless without access to KMS, next to that it makes it possible to version everything related to the application and removes manually entering passwords to SSM. @Erik Osterman (Cloud Posse) pointed out to me that git is about transparency and that storing anything else than readable text is a conflict. For me this is less of a problem when the alternative is to manually enter passwords. Happy to collect your thoughts.

Gonzalo Peciover 7 years ago

@Gonzalo Peci has joined the channel

jonboulleover 7 years ago

@jonboulle has joined the channel

Jamie Nelson (BitFlight)over 7 years ago

@Jamie Nelson (BitFlight) has joined the channel

jonboulleover 7 years ago

git isn't just about transparency in terms of content, it's about transparency in terms of an audit log of changes, which this still provides. of course you can get that with other systems too, but isn't the point of gitops to have all of that centralised in git? 🙃

Stevenover 7 years ago

@Steven has joined the channel

Erik Osterman (Cloud Posse)over 7 years ago

@jonboulle in principle I agree with you. but stoking debate, where is the line drawn? what about binary artifacts from build processes or packages?

Erik Osterman (Cloud Posse)over 7 years ago(edited)

@Maarten van der Hoef in the end, I don’t have a better solution than what you proposed. i still dislike binary assets in git, but like with images it’s a necessary tradeoff sometimes. i think it’s technically sound strategy and quite elegant if you get it working end-to-end.

Andriy Knysh (Cloud Posse)over 7 years ago

https://medium.com/airbnb-engineering/production-secret-management-at-airbnb-ad230e1bc0f6

jonboulleover 7 years ago

I don't think it's hard to draw the line; artifacts from build processes belong in blob stores (e.g. package repository or image repository). ideally the store is a CAS. then git can house a reference to them as necessary

jonboulleover 7 years ago

you'll find no disagreement from me on storing a lambda zip being an antipattern

jonboulleover 7 years ago

but to push back on "storing anything else than readable text is a conflict" (not sure if you directly said this or it's a paraphrase - so apologies for the potential strawman!) - there are plenty of cases of these kinds of content-addressable references (e.g. docker image SHA, Gemfile.lock, whatever) where they should definitely be stored in git but they can hardly qualify as "readable text" in themselves

Maarten van der Hoefover 7 years ago

@Andriy Knysh (Cloud Posse) thanks.. I responded to the article as this is a step away from factor12. But with their java background env variables were probably something they have never used to begin with.

Erik Osterman (Cloud Posse)over 7 years ago

@Maarten van der Hoef insightful comment

erikover 7 years ago

Erik Osterman (Cloud Posse)over 7 years ago

this is a good point

Erik Osterman (Cloud Posse)over 7 years ago

what is your approach to this in what you’re doing?

Erik Osterman (Cloud Posse)over 7 years ago(edited)

e.g. if you have the “widget” app and you encrypt the “prod” mailgun key and commit to source control in the widget app repo, is this blurring the line?

Dave Yuover 7 years ago

@Dave Yu has joined the channel

Erik Osterman (Cloud Posse)over 7 years ago

for those interested and want to follow along, we’re working on some enhancements for atlantis

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/atlantis/issues

Erik Osterman (Cloud Posse)over 7 years ago

we’ve added atlantis to our alpine repository:

Erik Osterman (Cloud Posse)over 7 years ago


curl -sSL <https://apk.cloudposse.com/install.sh> | sudo bash

Erik Osterman (Cloud Posse)over 7 years ago

And the work for integrating it into geodesic: https://github.com/cloudposse/geodesic/pull/298/files

antonbabenkoover 7 years ago

@antonbabenko has joined the channel

Rohit Vermaover 7 years ago

@Rohit Verma has joined the channel

erikover 7 years ago

Maarten van der Hoefover 7 years ago(edited)

AirBNB response to my question about having secrets coupled into Docker. Having secrets maintained away from the application makes it impossible to do an actual rollback to point an application to the previous config set which is a point.

I'm thinking of a way to make SSM/KMS/Chamber compatible with the mindset of that issue at stake.

SSM


/service/db_host  = 'hostA'
/service/db_user  = 'userA'
/service/db_host_migration = 'hostB'
/service/db_user_migration = 'userB'

On the Docker with Chamber /etc/chamber_rewrites ( or defined ENV ) defines how certain env vars can be rewritten with other created env vars.


db_host = db_host_migration
db_user = db_user_migration

This way it is possible to work with a default config set, yet it is still possible to fix config values to a certain deployment. Problem here is transparency, it is not visible from the outside which docker is using what.

Erik Osterman (Cloud Posse)over 7 years ago

re:


/service/db_host  = 'hostA'
/service/db_user  = 'userA'
/service/db_host_migration = 'hostB'
/service/db_user_migration = 'userB'

Erik Osterman (Cloud Posse)over 7 years ago

why not use different service namespaces

Erik Osterman (Cloud Posse)over 7 years ago

e.g.

Erik Osterman (Cloud Posse)over 7 years ago


/service/db_host  = 'hostA'
/service/db_user  = 'userA'
/service/migation/db_host = 'hostB'
/service/migration/db_user = 'userB'

Maarten van der Hoefover 7 years ago

TBH, I did think of that too, but I'm afraid that would kill their parser a bit.

Erik Osterman (Cloud Posse)over 7 years ago

their = chamber?

Maarten van der Hoefover 7 years ago

yes

Erik Osterman (Cloud Posse)over 7 years ago

hrm

Erik Osterman (Cloud Posse)over 7 years ago

true, i think their import/export only handles one service namespace at a time

Erik Osterman (Cloud Posse)over 7 years ago

checking

Maarten van der Hoefover 7 years ago

i mean, db_host is attribute here, migation/db_host is a nested attribute but has a / in it. I don't think / are allowed

Maarten van der Hoefover 7 years ago

inside the name

Maarten van der Hoefover 7 years ago

and a full real namespace change, as in, changing the prefix of everything SSM, or a different service name. That of course can work. BUT. you would need to modify iam policies for the service to allow access to it, and I think that is just too much change for something trivial.

erikover 7 years ago

Erik Osterman (Cloud Posse)over 7 years ago(edited)

anyways, what i was actually hoping was that the export format supported the service namespace

Erik Osterman (Cloud Posse)over 7 years ago

but it doesn’t

Erik Osterman (Cloud Posse)over 7 years ago

so to your point

Maarten van der Hoefover 7 years ago

I mean, by changing the full service path, and making it a level deeper more or less, you can make it to work but you would need to duplicate everything to that level deeper

Erik Osterman (Cloud Posse)over 7 years ago(edited)

i think a cool/needed tool would be a chamber ETL tool with KMS (PKE) support

Erik Osterman (Cloud Posse)over 7 years ago

that maybe supports some custom mapping

Erik Osterman (Cloud Posse)over 7 years ago

(since i don’t think the guys at chamber are too amenable to changes of this nature)

Maarten van der Hoefover 7 years ago

Maybe yes, but it's an interesting quest to drop and see what comes out.

Erik Osterman (Cloud Posse)over 7 years ago

despite my previous reservations, I think I am leaning still towards your approach

Maarten van der Hoefover 7 years ago

Better ( but impossible to get 🙂 ) is a new style SSM with versioned change-sets which can be referred to.

stephenover 7 years ago

@stephen has joined the channel

Adamover 7 years ago

@Adam has joined the channel

Yoannover 7 years ago

@Yoann has joined the channel

Erik Osterman (Cloud Posse)over 7 years ago

@antonbabenko @Maarten van der Hoef any companies you know of using atlantis that I can reference in my slides?

antonbabenkoover 7 years ago

I don’t remember names. They were pretty small and unknown.

Erik Osterman (Cloud Posse)over 7 years ago

Thanks anyways :)

Maarten van der Hoefover 7 years ago

Blinkist.com , need to ask the other two for OK

Erik Osterman (Cloud Posse)over 7 years ago

ok, great, I’ll add them - thanks!

Maarten van der Hoefover 7 years ago

When is it due?

Maarten van der Hoefover 7 years ago

https://www.reddit.com/r/devops/comments/9mpp0g/terraform_plugins_for_privileged_access/

antonbabenkoover 7 years ago

Send a email to Luke to get more names.

Erik Osterman (Cloud Posse)over 7 years ago

Talking 10/25

Erik Osterman (Cloud Posse)over 7 years ago

I also pinged Luke in their new slack community

benover 7 years ago

@ben has joined the channel

Gauravover 7 years ago

@Gaurav has joined the channel

Florian Drescherover 7 years ago

@Florian Drescher has joined the channel

Erik Osterman (Cloud Posse)over 7 years ago

I added a #atlantis channel since we're doing a lot more with it these days (related to runatlantis.io).

Andriy Knysh (Cloud Posse)over 7 years ago

continuing this discussion https://sweetops.slack.com/archives/CBW0HJDS8/p1539711018000100

Andriy Knysh (Cloud Posse)over 7 years ago

nice article https://css-tricks.com/introducing-github-actions/

Andriy Knysh (Cloud Posse)over 7 years ago

they will have a service to add encrypted secrets

Andriy Knysh (Cloud Posse)over 7 years ago

you could use ENV vars

Andriy Knysh (Cloud Posse)over 7 years ago

and you’ll be able to create actions programmatically

Andriy Knysh (Cloud Posse)over 7 years ago

workflow "New workflow" {
  on = "push"
  resolves = ["Deploy to Azure"]
}

action "Deploy to Azure" {
  uses = "./.github/azdeploy"
  secrets = ["SERVICE_PASS"]
  env = {
    SERVICE_PRINCIPAL="<http://sdrasApp>",
    TENANT_ID="72f988bf-86f1-41af-91ab-2d7cd011db47",
    APPID="sdrasMoonshine"
  }
}

Andriy Knysh (Cloud Posse)over 7 years ago

looks like Terraform HCL 🙂

Erik Osterman (Cloud Posse)over 7 years ago

Yea that’s 100% :-)

September 2018 Browse by date January 2019