#release-engineering - April 2020

noooooooooooo the thread about atlantis is gone!!!!!!!!

the first thread…. wasn’t there an archive ?

I found it….

does someone here have a tool for a timeline ?

Currently we are using slack and a gitlab snippet as a log of when we updated certain components but I think it would be nice to have a tool that allows you to say we have updated this and just go back in time?

Also it should maybe be able to aggregate other sources like alert manager and bugsnag.

Hi there! I am trying to write scripts to reset database states so that some tests can be done. Are there any good tools for managing that?

Also trying to figure out if there are good tools for anonymizing data from prod. Thank you.

Possibly liquibase or Flyway. I've only used Flyway before via Maven though. Its more involved than running a simple script though. The clean command would 'reset' things

Data anonymizing is highly dependent upon your schema and so I'm thinking you are going to be doing manual scripting. Knowing nothing about your database there are a few tools out there worth taking a peek at, https://github.com/davedash/mysql-anonymous or http://sunitparekh.github.io/data-anonymization/ are a few

does someone have a semver check regex handy that works in bash.?

I am trying to re-do one of my test environment to allow testing SMS and email. Rather than sending real SMS and email, any best practices for how to set this up. Should we mock them or are there 3rd party services we can use for this?

@Adam Watson is this something you have worked with? ☝️

@Adam Watson has joined the channel

I generally mock them for testing, for Twilio which we use there is also an option to use an API token earmarked for development for billing purposes.

We also keep dev/qa/staging/training accounts and have special phone numbers isolated to those environments

I'd also be interested if you come across third party services 🙂.

@Adam Watson Yes we are on Twilio as well. In general, we just have a prod and a non-prod acct. We don’t further differentiate like you have done. Any special advantage for doing that?

Mostly for end users, we do a lot with Flex and between QA, Training, and all the other teams it makes life easier.

Often staging different plugins for the UI, or various systems, or things like tearing a back office system down each night on a training space for call centers etc.

ICT/FAT/UAT... we have a lot of hoops to jump through. 😫

Ok that makes sense. Thank you for your input

dunno the best place to put this, but currently working on a release pipeline. for shell gurus out there, is there an easy way to dynamically parse like env vars into json config file i.e.

{
 "foo": "${BAR}",
 "asdf": "${ASDF}",
 "apple": "${PIE}",
}

There’s a command called envsubst that ships with most Linux distros

It can handle exactly the example you provide above

i second to it either envsubst or some places i do sed replacement

and the different json files can have any env variables?

whoops 🙂

well that is a snippet of how you might use it to selectively replace variables

not super elegant or as fun as gomplate but it gets the job done.

wow this is the perfect tool 👌👌

After a while, you'll probably reach a point where envsubst is not enough

That's when you can use https://gomplate.ca/

note that gomplate let's you load settings from files, SSM, http, and probably a dozen different places as well as use logic in the form of conditionals

like parameter store ?

Looks like it: https://docs.gomplate.ca/datasources/#using-aws-smp-datasources

I'll go as far as to recommend skipping over envsubst and using something like gomplate to start with. That way you aren't converting template files later on to the tooling you end up eventually switching over to when you get sick of bash hacks (whether it ends up being gomplate or dockerize or anything else that is way more capable than envsubst).

the double-whammy is that you are then also practicing the templating language that helm uses (with some of its own special functions)